Skip to content

Commit 26da1bf

Browse files
committed
[FLINK-22373][docs] Add Flink 1.13 release notes
This closes #15687
1 parent 56c61dd commit 26da1bf

File tree

3 files changed

+565
-1
lines changed

3 files changed

+565
-1
lines changed
Lines changed: 282 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,282 @@
1+
---
2+
title: "Release Notes - Flink 1.13"
3+
---
4+
<!--
5+
Licensed to the Apache Software Foundation (ASF) under one
6+
or more contributor license agreements. See the NOTICE file
7+
distributed with this work for additional information
8+
regarding copyright ownership. The ASF licenses this file
9+
to you under the Apache License, Version 2.0 (the
10+
"License"); you may not use this file except in compliance
11+
with the License. You may obtain a copy of the License at
12+
13+
http://www.apache.org/licenses/LICENSE-2.0
14+
15+
Unless required by applicable law or agreed to in writing,
16+
software distributed under the License is distributed on an
17+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
18+
KIND, either express or implied. See the License for the
19+
specific language governing permissions and limitations
20+
under the License.
21+
-->
22+
23+
# Release Notes - Flink 1.13
24+
25+
These release notes discuss important aspects, such as configuration, behavior, or dependencies,
26+
that changed between Flink 1.12 and Flink 1.13. Please read these notes carefully if you are
27+
planning to upgrade your Flink version to 1.13.
28+
29+
### Failover
30+
31+
#### Remove state.backend.async option.
32+
33+
##### [FLINK-21935](https://issues.apache.org/jira/browse/FLINK-21935)
34+
35+
The `state.backend.async` option is deprecated. Snapshots are always asynchronous now (as they were
36+
by default before) and there is no option to configure a synchronous snapshot any more.
37+
38+
The constructors of `FsStateBackend` and `MemoryStateBackend` that take a flag for sync/async
39+
snapshots are kept for API compatibility, but the flags are ignored now.
40+
41+
#### Unify binary format for Keyed State savepoints
42+
43+
##### [FLINK-20976](https://issues.apache.org/jira/browse/FLINK-20976)
44+
45+
Flink’s savepoint binary format is unified across all state backends. That means you can take a
46+
savepoint with one state backend and then restore it using another.
47+
48+
If you want to switch the state backend you should first upgrade your Flink version to 1.13, then
49+
take a savepoint with the new version, and only after that, you can restore it with a different
50+
state backend.
51+
52+
#### FailureRateRestartBackoffTimeStrategy allows one less restart than configured
53+
54+
##### [FLINK-20752](https://issues.apache.org/jira/browse/FLINK-20752)
55+
56+
The Failure Rate Restart Strategy was allowing 1 less restart per interval than configured. Users
57+
wishing to keep the current behavior should reduce the maximum number of allowed failures per
58+
interval by 1.
59+
60+
#### Support rescaling for Unaligned Checkpoints
61+
62+
##### [FLINK-17979](https://issues.apache.org/jira/browse/FLINK-17979)
63+
64+
While recovering from unaligned checkpoints, users can now change the parallelism of the job. This
65+
change allows users to quickly upscale the job under backpressure.
66+
67+
### SQL
68+
69+
#### Officially deprecate the legacy planner
70+
71+
##### [FLINK-21709](https://issues.apache.org/jira/browse/FLINK-21709)
72+
73+
The old planner of the Table & SQL API is deprecated and will be dropped in Flink 1.14. This means
74+
that both the BatchTableEnvironment and DataSet API interop are reaching end of life. Use the
75+
unified TableEnvironment for batch and stream processing with the new planner, or the DataStream API
76+
in batch execution mode.
77+
78+
#### Use TIMESTAMP_LTZ as return type for function PROCTIME()
79+
80+
##### [FLINK-21714](https://issues.apache.org/jira/browse/FLINK-21714)
81+
82+
Before Flink 1.13, the function return type of `PROCTIME()` is `TIMESTAMP`, and the return value is
83+
the `TIMESTAMP` in UTC time zone, e.g. the wall-clock shows `2021-03-01 12:00:00` at Shanghai,
84+
however the `PROCTIME()` displays `2021-03-01 04:00:00` which is wrong. Flink 1.13 fixes this issue
85+
and uses `TIMESTAMP_LTZ` type as return type of `PROCTIME()`, users don't need to deal time zone
86+
problems anymore.
87+
88+
#### Support defining event time attribute on TIMESTAMP_LTZ column
89+
90+
##### [FLINK-20387](https://issues.apache.org/jira/browse/FLINK-20387)
91+
92+
Support defining event time attribute on TIMESTAMP_LTZ column, base on this, Flink SQL gracefully
93+
support the Daylight Saving Time.
94+
95+
#### Correct function CURRENT_TIMESTAMP/CURRENT_TIME/CURRENT_DATE/LOCALTIME/LOCALTIMESTAMP/NOW()
96+
97+
##### [FLINK-21713](https://issues.apache.org/jira/browse/FLINK-21713)
98+
99+
The value of time function CURRENT_TIMESTAMP and NOW() are corrected from UTC time with `TIMESTAMP`
100+
type to epoch time with `TIMESTAMP_LTZ` type. Time function LOCALTIME, LOCALTIMESTAMP, CURRENT_DATE,
101+
CURRENT_TIME, CURRENT_TIMESTAMP and NOW() are corrected from evaluates for per record in batch mode
102+
to evaluate once at query-start for batch job.
103+
104+
#### Disable problematic cast conversion between NUMERIC type and TIMESTAMP type
105+
106+
##### [FLINK-21698](https://issues.apache.org/jira/browse/FLINK-21698)
107+
108+
The CAST operation between `NUMERIC` type and `TIMESTAMP` type is problematic and is disabled now,
109+
e.g.
110+
`CAST(numeric AS TIMESTAMP(3))` is disabled and should use `TO_TIMESTAMP(FROM_UNIXTIME(numeric))`
111+
instead.
112+
113+
#### Support USE MODULES syntax
114+
115+
##### [FLINK-21298](https://issues.apache.org/jira/browse/FLINK-21298)
116+
117+
The term MODULES is a reserved keyword now. Use backticks to escape column names and other
118+
identifiers with this name.
119+
120+
#### Update TableResult.collect()/TableResult.print() to the new type system
121+
122+
##### [FLINK-20613](https://issues.apache.org/jira/browse/FLINK-20613)
123+
124+
`Table.execute().collect()` might return slightly different results for column types and row kind.
125+
The most important differences include:
126+
127+
* Structured types are represented as POJOs of the original class and not Row anymore.
128+
* Raw types are serialized according to the configuration in TableConfig.
129+
130+
131+
#### Update the Row.toString method
132+
133+
##### [FLINK-18090](https://issues.apache.org/jira/browse/FLINK-18090)
134+
135+
The `Row.toSting()` method has been reworked. This is an incompatible change. If the legacy
136+
representation is still required for tests, the old behavior can be restored via the flag
137+
`RowUtils.USE_LEGACY_TO_STRING` for the local JVM. However, relying on the row's string
138+
representation for tests is not a good idea in general as field data types are not verified.
139+
140+
#### Support start SQL Client with an initialization SQL file
141+
142+
##### [FLINK-20320](https://issues.apache.org/jira/browse/FLINK-20320)
143+
144+
The `sql-client-defaults.yaml` YAML file is deprecated and not provided in the release package. To
145+
be compatible, it's still supported to initialize the SQL Client with the YAML file if manually
146+
provided. But it's recommend to use the new introduced `-i` startup option to execute an
147+
initialization SQL file to setup the SQL Client session. The so-called initialization SQL file can
148+
use Flink DDLs to define available catalogs, table sources and sinks, user-defined functions, and
149+
other properties required for execution and deployment. The support of legacy SQL Client YAML file
150+
will be totally dropped in Flink 1.14.
151+
152+
#### Hive dialect no longer supports Flink syntax for DML and DQL
153+
154+
##### [FLINK-21808](https://issues.apache.org/jira/browse/FLINK-21808)
155+
156+
Hive dialect supports HiveQL for DML and DQL. Please switch to default dialect in order to write in
157+
Flink syntax.
158+
159+
### Runtime
160+
161+
#### BoundedOneInput.endInput is called when taking synchronous savepoint
162+
163+
##### [FLINK-21132](https://issues.apache.org/jira/browse/FLINK-21132)
164+
165+
`endInput()` is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is
166+
stopping with savepoint.
167+
168+
#### Remove JobManagerOptions.SCHEDULING_STRATEGY
169+
170+
##### [FLINK-20591](https://issues.apache.org/jira/browse/FLINK-20591)
171+
172+
The configuration parameter `jobmanager.scheduler.scheduling-strategy` has been removed, because
173+
the `legacy` scheduler has been removed from Flink 1.13.0.
174+
175+
#### Warn user if System.exit() is called in user code
176+
177+
##### [FLINK-15156](https://issues.apache.org/jira/browse/FLINK-15156)
178+
179+
A new configuration value `cluster.intercept-user-system-exit` allows to log a warning, or throw an
180+
exception if user code calls `System.exit()`.
181+
182+
This feature is not covering all locations in Flink where user code is executed. It just adds the
183+
infrastructure for such an interception. We are tracking this improvement in
184+
[FLINK-21307](https://issues.apache.org/jira/browse/FLINK-21307).
185+
186+
### Docker
187+
188+
#### Consider removing automatic configuration fo number of slots from docker
189+
190+
##### [FLINK-21036](https://issues.apache.org/jira/browse/FLINK-21036)
191+
192+
The docker images no longer set the default number of taskmanager slots to the number of CPU cores.
193+
This behavior was inconsistent with all other deployment methods and ignored any limits on the CPU
194+
usage set via docker.
195+
196+
#### Rework jemalloc switch to use an environment variable
197+
198+
##### [FLINK-21034](https://issues.apache.org/jira/browse/FLINK-21034)
199+
200+
The docker switch for disabling the jemalloc memory allocator has been reworked from a script
201+
argument to an environment variable called DISABLE_JEMALLOC. If set to "true" jemalloc will not be
202+
enabled.
203+
204+
### Connectors
205+
206+
#### Remove swift FS filesystem
207+
208+
##### [FLINK-21819](https://issues.apache.org/jira/browse/FLINK-21819)
209+
210+
The Swift filesystem is no longer being actively developed and has been removed from the project and
211+
distribution.
212+
213+
##### [FLINK-22133](https://issues.apache.org/jira/browse/FLINK-22133)
214+
The unified source API for connectors has a minor breaking change. The `SplitEnumerator.snapshotState()`
215+
method was adjusted to accept the *Checkpoint ID* of the checkpoint for which the snapshot is created.
216+
217+
### Monitoring & debugging
218+
219+
#### Introduce latency tracking state
220+
221+
##### [FLINK-21736](https://issues.apache.org/jira/browse/FLINK-21736)
222+
223+
State access latency metrics are introduced to track all kinds of keyed state access to help debug
224+
state performance. This feature is not enabled by default and can be turned on by
225+
setting `state.backend.latency-track.keyed-state-enabled` to true.
226+
227+
#### Support for CPU flame graphs in web UI
228+
229+
##### [FLINK-13550](https://issues.apache.org/jira/browse/FLINK-13550)
230+
231+
Flink now offers flame graphs for each node in the job graph. Please enable this experimental feature
232+
by setting the respective configuration flag `rest.flamegraph.enabled`.
233+
234+
#### Display last n exceptions/causes for job restarts in Web UI
235+
236+
##### [FLINK-6042](https://issues.apache.org/jira/browse/FLINK-6042)
237+
238+
Flink exposes the exception history now through the REST API and the UI. The amount of most-recently
239+
handled exceptions that shall be tracked can be defined through `web.exception-history-size`.
240+
Some values of the exception history's REST API Json response are deprecated as part of this effort.
241+
242+
#### Create backPressuredTimeMsPerSecond metric
243+
244+
##### [FLINK-20717](https://issues.apache.org/jira/browse/FLINK-20717)
245+
246+
Previously `idleTimeMsPerSecond` was defined as the time task spent waiting for either the input or
247+
the back pressure. Now `idleTimeMsPerSecond` excludes back pressured time, so if the task is back
248+
pressured it is not idle. The back pressured time is now measured separately
249+
as `backPressuredTimeMsPerSecond`.
250+
251+
#### Enable log4j2 monitor interval by default
252+
253+
##### [FLINK-20510](https://issues.apache.org/jira/browse/FLINK-20510)
254+
255+
The Log4j support for updating the Log4j configuration at runtime has been enabled by default. The
256+
configuration files are checked for changes every 30 seconds.
257+
258+
#### ZooKeeper quorum fails to start due to missing log4j library
259+
260+
##### [FLINK-20404](https://issues.apache.org/jira/browse/FLINK-20404)
261+
262+
The Zookeeper scripts in the Flink distribution have been modified to disable the Log4j JMX
263+
integration due to an incompatibility between Zookeeper 3.4 and Log4j 2. To re-enable this feature,
264+
remove the line in the `zookeeper.sh` file that sets `zookeeper.jmx.log4j.disable`.
265+
266+
#### Expose stage of task initialization
267+
268+
##### [FLINK-17012](https://issues.apache.org/jira/browse/FLINK-17012)
269+
270+
Task's RUNNING state was split into two states: INITIALIZING and RUNNING. Task is INITIALIZING while
271+
state is initialising and in case of unaligned checkpoints, until all the in-flight data has been
272+
recovered.
273+
274+
### Deployment
275+
276+
#### Officially deprecate Mesos support
277+
278+
##### [FLINK-22352](https://issues.apache.org/jira/browse/FLINK-22352)
279+
280+
The community decided to deprecate the Apache Mesos support for Apache Flink. It is subject to
281+
removal in the future. Users are encouraged to switch to a different resource manager.
282+

docs/content/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,6 @@ Release notes cover important changes between Flink versions. Please read them c
8383
For some reason Hugo will only allow linking to the
8484
release notes if there is a leading '/' and file extension.
8585
-->
86-
See the release notes for [Flink 1.12]({{< ref "/release-notes/flink-1.12.md" >}}), [Flink 1.11]({{< ref "/release-notes/flink-1.11.md" >}}), [Flink 1.10]({{< ref "/release-notes/flink-1.10.md" >}}), [Flink 1.9]({{< ref "/release-notes/flink-1.9.md" >}}), [Flink 1.8]({{< ref "/release-notes/flink-1.8.md" >}}), or [Flink 1.7]({{< ref "/release-notes/flink-1.7.md" >}}).
86+
See the release notes for [Flink 1.13]({{< ref "/release-notes/flink-1.13.md" >}}), [Flink 1.12]({{< ref "/release-notes/flink-1.12.md" >}}), [Flink 1.11]({{< ref "/release-notes/flink-1.11.md" >}}), [Flink 1.10]({{< ref "/release-notes/flink-1.10.md" >}}), [Flink 1.9]({{< ref "/release-notes/flink-1.9.md" >}}), [Flink 1.8]({{< ref "/release-notes/flink-1.8.md" >}}), or [Flink 1.7]({{< ref "/release-notes/flink-1.7.md" >}}).
8787

8888
{{< /columns >}}

0 commit comments

Comments
 (0)