Vitess VReplication

VReplication 

Açıklaması şöyle. Yani advanced replication içindir.  Örneğin Re-sharding için kullanılabilir

VReplication is a low level feature in Vitess that enables creating one or more VStreams. 

Bileşenleri şöyle 

1. VStream

2. VStreamer

The story of Vitess yazısında çok bilgi var

Her şeyi gösteren şekil şöyle. Benzer bir şekil burada

1. VStream

Açıklaması şöyle. VTGate üzerinde çalışır. Binlog event'lerine benzer bilgi taşır

VTGate exposes a gRPC service called VStream. It is a server-side streaming service. Any gRPC client can subscribe to the VStream service to get a continuous stream of change events from the underlying MySQL instances. The change events that VStream emits have similar information to the MySQL binary logs of the underlying MySQL instances. A single VStream can even subscribe to multiple shards for a given keyspace, making it quite a convenient API to build CDC tools.

2. VStreamer

Açıklaması şöyle. VTTablet üzerinde çalışır.

VTTablet host runs a component called VStreamer. VStreamer fetches events from the MySQL bin log and makes them accessible to other components upstream of VTTablet.

So lets say a write arrives at a VTTablet host, the following will happen

- The write will be recorded to MySQL

- As part of persisting this write the write will also be recored in the MySQL bin log (more details on MySQL bin log here).

- VStreamer will fetch updates from the bin log and expose those events for upstream components to consume.

Scaling VReplication

Scaling VReplication yazısında açıklaması şöyle

What is VReplication? To sum its docs, it’s a tool that lets you copy and maintain a live subset of data from one database to another – similar to Shopify’s Ghostferry, but tailored for the Vitess’s world and with an orchestration layer on top of it.

VReplication is a core component of Vitess that backs many of its features, but where does its code actually run?

- Whenever you create a VReplication stream by executing a SQL statement like INSERT INTO _vt.vreplication (db_name, source, pos, ...), VTTablet that is the destination of the stream will create multiple goroutines per each stream:

- A goroutine to manage the stream. This one will stop replication if you update the stream state to “Stopped” externally

- A goroutine to fetch binlog events from source over GRPC

- A goroutine to apply fetched binlog events

- A few goroutines to manage HTTP2/GRPC connection

- Full dump of goroutines

Schema şöyle

Tablolar şöyle

vreplication tablosundaki sütunlar şöyle. Bu tablolara vtgate aracılığıyla dışarıdan erişemedim.