18 Haziran 2023 Pazar

Debezium Connector transforms - ExtractNewRecordState - Sadece Değişen Alan Gelir

Giriş
Şu iki satırı yazmak gerekir.
1. transforms": "unwrap"
2. "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
Böylece çıktıda  before alanı hep null gelir. Yani şöyledir
{
"source": { ... }, "before": null, "after": { "id": 1, "name": "John Doe", "age": 30 }, "op": "c", "ts_ms": 1654316585000 }
Açıklaması şöyle
By default, Debezium sends all events in an envelope that includes many pieces of information about the change captured. I’m only interested in reading the changed value here, so the command tells Kafka Connect to keep this information and discard the rest.
Örnek
Şöyle yaparız
{
  "name":"customer-connector",
  "config":{
   "connector.class":"io.debezium.connector.mysql.MySqlConnector",
   "tasks.max":"1",
   "database.hostname":"localhost",
   "database.port":"3306",
   "database.user":"root",
   "database.password":"mysqlpwd",
   "database.server.id":"184054",
   "database.server.name":"mysql",
   "database.whitelist":"customers_db",
   "database.history.kafka.bootstrap.servers":"localhost:9092",
   "database.history.kafka.topic":"schema-changes.customers",
   "transforms":"unwrap",    "transforms.unwrap.type":"io.debezium.transforms.ExtractNewRecordState",
    "transforms.unwrap.drop.tombstones":false,
    "transforms.unwrap.delete.handling.mode":"rewrite"
   }
}

12 Haziran 2023 Pazartesi

Gap Lock ve Next-key Lock

1. Gap Lock
Açıklaması şöyle. Sanırım sadece Repeatable Read seviyesinde Gap Lock var
A gap lock is a shared lock that locks the gaps between two existing index keys in a table.

If a gap lock is held, no other statement is allowed to INSERT, UPDATE, or DELETE a record that would fall within that gap.

This prevents phantom reads and ensures that a range of values remains consistent throughout the lifetime of a transaction.
Açıklaması şöyle
Here are two scenarios where a gap lock is acquired automatically.

- Range query. When a transaction queries a range of records.
- Index query using shared or exclusive lock. InnoDB automatically acquires gap locks on the gap before and after the indexed record. (Also known as next-key lock)
Örnek
Başlarken şöyle olsun
# Confirm the database version.
mysql> select @@version;
+-----------+
| @@version |
+-----------+
| 5.7.30    |
+-----------+
1 row in set (0.00 sec)

# Confirm the database isolation level.
mysql> select @@transaction_isolation;
+-------------------------+
| @@transaction_isolation |
+-------------------------+
| REPEATABLE-READ         |
+-------------------------+
1 row in set (0.00 sec)

# Turn off autocommit for MySQL.
mysql> select @@autocommit;
+--------------+
| @@autocommit |
+--------------+
|            1 |
+--------------+
1 row in set (0.01 sec)

mysql> set autocommit=0;
Query OK, 0 rows affected (0.00 sec)
Elimizde şöyle bir tablo olsun. age sütununda index var. Yaşlar 10 ve 20
CREATE TABLE `students` (
  `id` int NOT NULL AUTO_INCREMENT,
  `age` int DEFAULT 0,
  `name` varchar(64) DEFAULT '',
  PRIMARY KEY (`id`),
  KEY `idx_age` (`age`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=1;

# insert data
mysql> insert into students(`age`, `name`) values (10, 'foo'), (20, 'bar');
Query OK, 2 rows affected (0.00 sec)
Records: 2  Duplicates: 0  Warnings: 0

mysql> commit;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from students;
+----+------+------+
| id | age  | name |
+----+------+------+
|  1 |   10 | foo  |
|  2 |   20 | bar  |
+----+------+------+
2 rows in set (0.00 sec)
İki tane transaction başlatalım 
TA (10, +∞) için lock koyar. 
TB (-∞, 20) için lock koyar. 
# TA
mysql> begin;

mysql> update students set name = 'bar_a' where age = 20;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

# TB
mysql> begin;

mysql> update students set name = 'foo_b' where age = 10;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0
TA bloke olur, çünkü TB'nin alanına girmektedir
mysql> insert into students(`age`, `name`) values (15, 'lee');
|
TB de bloke olur ancak bir müddet sonra TA rollback yaptığı için devam edebilir
mysql> insert into students(`age`, `name`) values (30, 'dec');
Query OK, 1 row affected (0.00 sec)
Dead lock loglarına bakmak için şöyle yaparız
show engine innodb status;


2. Next-key Lock
Açıklaması şöyle
Similar to gap locks, next-key locks prevent phantom reads.

A next-key lock is a combination of two different locks
- A record lock (shared or exclusive lock)
- A gap lock

When a read statement is executed via a non-clustered index with a shared or exclusive lock, the database acquires
- A record lock on the record
- Gap locks on the gap before and after the indexed record.

This ensures that no new rows can be inserted before and after the indexed entry, ensuring data consistency.
Açıklaması şöyle
But here’s the intriguing part, the behaviour of a next-key lock varies depending on the type of index being used:

Querying using a non-clustered index
The database acquires a record lock and gap locks on the gaps before and after the index entry.

Querying using a clustered index
The database acquires only a record lock without a gap lock.

Querying using a non-indexed column
The database acquires a record lock and a gap lock on the ENTIRE table.
As a result, querying a record using a non-indexed column prevents other transactions from modifying any records in the table.

Örnek - non-clustered index
Elimizde şöyle bir tablo olsun. Burada non-clustered index kullanıldığı için bir önceli ve bir sonraki kayıt ta kilitlendi
/* Sample table */
+---------+---------+------+-------------------+
| id      | user_id | age  | email             |
+---------+---------+------+-------------------+
| 1000076 |     101 |   12 | peter@hotmail.com |
| 1000080 |     104 |   17 | dan@hotmail.com   |
| 1000079 |     107 |   19 | lim@hotmail.com   |
| 1000081 |     109 |   20 | chris@hotmail.com |
+---------+---------+------+-------------------+
Tabloya select for update yapalım
/* Session 1 */
mysql> begin;

mysql> select * from user_tab where user_id=107 for update;
+---------+---------+------+-----------------+
| id      | user_id | age  | email           |
+---------+---------+------+-----------------+
| 1000079 |     107 |   19 | lim@hotmail.com |
+---------+---------+------+-----------------+

/* Session 2 */
mysql> begin;

mysql> insert into user_tab (user_id, age, email) 
       values (106, 10, 'lee@gmail.com');
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

mysql> insert into user_tab (user_id, age, email) 
       values (108, 10, 'lee@gmail.com');
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

mysql> insert into user_tab (user_id, age, email) 
       values (110, 10, 'lee@gmail.com');
Query OK, 1 row affected (0.00 sec)
Açıklaması şöyle
In the example above, the first transaction queried a record using an xLock. The database locks the record and also the gaps from user_id=104 to user_id=109.

Hence, the second transaction was blocked when trying to insert a record within the gap user_id=104-109. When trying to insert a record outside of the gap user_id=110, the statement succeeded.


Select For Share - Read Lock

Giriş
Açıklaması şöyle
Shared lock, also known as a read lock, allows multiple transactions to read the same record concurrently.

Transactions that acquire a shared lock can only read but not modify the data. It’s like a “no harm done” situation, where no transaction can modify the data while others are reading it.

If other transactions attempt to modify the record, they will be blocked until all shared locks are released.
Örnek
Şöyle yaparız
/* session 1 acquires a shared lock */
mysql> begin;

mysql> select * from user_tab where user_id=101 for share;
+---------+---------+------+-------------------+
| id      | user_id | age  | email             |
+---------+---------+------+-------------------+
| 1000076 |     101 |   10 | peter@hotmail.com |
+---------+---------+------+-------------------+


/* session 2 is blocked */
mysql> begin;

mysql> update user_tab set age=11 where user_id=101;
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction


7 Haziran 2023 Çarşamba

JDBC Sürücüleri - MySQL

Giriş
Açıklaması şöyle
The coordinates of the MySQL JDBC driver have changed from mysql:mysql-connector-java to com.mysql:mysql-connector-j.
Daha uzun bir açıklama şöyle
MySQL Connector/J has new Maven Coordinates
For a very long time MySQL Connector/J has been published under the Maven coordinates group ID 'mysql' and artifact ID 'mysql-connector-java'. Neither was fully compliant or been officially ratified.

While a group ID mysql was originally possible, nowadays it would be very difficult to obtain approval to use a single word group ID. Those who still have them are often considered legacy projects. Although Maven does not enforce this rule, the convention says that a group ID should follow Java's package name rules, i.e., it should start with a reversed domain name belonging to the organization that owns the project. For MySQL Connector/J this means it should have been com.mysql all along.

Another unfortunate naming choice was mysql-connector-java for artifact ID. It would have been all fine if only the product was officially named "MySQL Connector/Java" instead of "MySQL Connector/J". The name "connector-java" or "Connector/Java", with or without the preceding word "MySQL" has never considered official by the MySQL Organization. Again, Maven does not enforce any strict rules for naming artifacts but we feel that we should align the artifact ID with the product name so, mysql-connector-j, would have been a better match.

Having said that, starting with version 8.0.31, MySQL Connector/J artifacts can be found under the maven repository directory /com/mysql/mysql-connector-j, with group ID com.mysql and artifact ID mysql-connector-j. Consequently, the corresponding jar files were also renamed to mysql-connector-j-x.y.z.jar in all available distribution packages.

How does this affect your projects?
Projects with a dependency on MySQL Connector/J version 8.0.31 and above, must update their POM dependency settings to:

<dependency>
  <groupId>com.mysql</groupId>
  <artifactId>mysql-connector-j</artifactId>
  <version>8.0.31</version>
  (...)
</dependency>
 
For a limited number of releases the MySQL Connector/J development team will maintain the old Maven coordinates alongside with the new. The old repository will stop getting jar files, though. Instead, only informational files and a relocation POM file will be published under this repository and projects linking to the old Maven coordinates will be redirected to the new ones automatically by the dependency resolution tool of your choice.

Why now?
No reason at all. But why not? In fact there's no publicly visible reason to do this change now. We had the chance to fine tuning some company-internal publishing processes and this was just in the way. The impact for users is not negligible but in the end we just want to do the right thing.

Eski Kullanım
Örnek
Şöyle yaparız
<dependency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
  <version>8.0.27</version>
</dependency>
Örnek
Şöyle yaparız. Burada scope runtime veriliyor. Yani derleme için gerekmez ancak çalışmak için gerekir.
<dependency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
  <version>8.0.27</version>
  <scope>runtime</scope>
</dependency>

Soft Delete

Giriş Açıklaması  şöyle When using the soft delete mechanism on the database, you might run into a situation where a record with a unique co...