Add support for L1 to L2 message log syncing #2251

t00ts · 2024-09-19T10:53:02Z

Introduces functionality to sync L1 to L2 message logs, improving our ability to track cross-chain interactions.

Key Changes

Added get_logs method to the Ethereum client:
- Allows fetching logs for a specified block range
- Implements recursive fetching with automatic range splitting for large queries
Implemented bulk syncing of L1ToL2MessageLog:
- Fetches logs from the last synced L1 block up to the current state
- Stores these logs for later matching with L2 transactions
Enhanced L1 sync process:
- Bulk fetches logs at the beginning of execution
- Waits for L2 blocks and matches l1_handler transactions with corresponding L1 transaction hashes

CHANGELOG.md

crates/common/src/lib.rs

crates/ethereum/src/lib.rs

crates/storage/src/schema/revision_0065.rs

crates/pathfinder/src/state/sync.rs

…nd `L1ToL2MessageLog`s

kkovaacs

I've added a quite a few comments, most of them in these categories:

using structured fields in tracing macros: https://docs.rs/tracing/latest/tracing/index.html#recording-fields
we should be adding a newtype for L1 block number because we seem to be mixing L1/L2 block numbers at a few places
using storage row accessors and type conversions for L1 block number / L1 block hash instead of ad-hoc conversions in storage code

crates/ethereum/src/lib.rs

crates/pathfinder/src/state/sync.rs

kkovaacs · 2024-09-24T07:06:13Z

crates/pathfinder/src/state/sync.rs

@@ -686,8 +718,42 @@ async fn consumer(
                }
            }
            L1ToL2Message(msg) => {
-                tracing::trace!("Got a new L1 to L2 message log: {:?}", msg);
-                // todo!()
+                tracing::debug!("Got an L1ToL2Message: {:?}", msg.message_hash);


Suggested change

tracing::debug!("Got an L1ToL2Message: {:?}", msg.message_hash);

tracing::trace!(message_hash=%msg.message_hash, "Got an L1ToL2Message");

kkovaacs · 2024-09-24T07:07:23Z

crates/pathfinder/src/state/sync.rs

+                        ..
+                    }) = tx.fetch_l1_to_l2_message_log(&msg.message_hash)?
+                    {
+                        tracing::debug!("Found L2 tx for L1 Tx {:?}", l1_tx_hash);


Suggested change

tracing::debug!("Found L2 tx for L1 Tx {:?}", l1_tx_hash);

tracing::trace!(%l1_tx_hash, %l2_tx_hash, "Found L2 tx for L1 tx);

kkovaacs · 2024-09-24T07:07:47Z

crates/pathfinder/src/state/sync.rs

+                    }
+                    // Otherwise, we insert the message log with an empty L2 tx hash
+                    else {
+                        tracing::debug!("L2 tx NOT found for L1 Tx {:?}", l1_tx_hash);


Suggested change

tracing::debug!("L2 tx NOT found for L1 Tx {:?}", l1_tx_hash);

tracing::trace!(%l1_tx_hash, "L2 tx NOT found for L1 tx");

kkovaacs · 2024-09-24T08:17:03Z

crates/storage/src/connection/message.rs

+        let raw_data = stmt
+            .query_row(params![&message_hash.as_bytes().to_vec()], |row| {
+                Ok((
+                    row.get::<_, Option<u64>>(0)?,


Suggested change

row.get::<_, Option<u64>>(0)?,

row.get_optional_block_number(0)?,

This is an L1 block number so we should just use an the newtype for L1 block numbers.

kkovaacs · 2024-09-24T08:19:01Z

crates/storage/src/connection/message.rs

+            let debug_tx_str = match (&data.1, &data.2) {
+                (Some(_), None) => "[L1, X]",
+                (None, Some(_)) => "[X, L2]",
+                _ => "N/A",
+            };
+            tracing::debug!(
+                "Fetched (and found: {}) an L1 to L2 message log for {:?}",
+                debug_tx_str,
+                message_hash
+            );


Suggested change

let debug_tx_str = match (&data.1, &data.2) {

(Some(_), None) => "[L1, X]",

(None, Some(_)) => "[X, L2]",

_ => "N/A",

};

tracing::debug!(

"Fetched (and found: {}) an L1 to L2 message log for {:?}",

debug_tx_str,

message_hash

);

tracing::trace!(

?message_hash,

l1_tx_hash=?data.1,

l2_tx_hash=%data.2,

"Fetched an L1 to L2 message log"

);

kkovaacs · 2024-09-24T08:19:49Z

crates/storage/src/connection/message.rs

+            )
+            .context("Removing L1 to L2 message log")?;
+
+        tracing::trace!("Removed L1 to L2 message log: {:?}", message_hash);


Suggested change

tracing::trace!("Removed L1 to L2 message log: {:?}", message_hash);

tracing::trace!(?message_hash, "Removed L1 to L2 message log);

kkovaacs · 2024-09-24T08:29:41Z

crates/pathfinder/src/state/sync.rs

+            }
+            _ => BlockNumber::new_or_panic(0),
+        };
+        Ok(l1_handler_block.unwrap_or(default_block_number))


Hmm, I'm not sure I follow. hightest_block_with_l1_handler_tx() gets us the L1 block number for the latest L1 block where we have already found a matching L1 handler transaction in an L2 block.

I think that's a bit more pessimistic than what we'd need here. Ideally we should be just making sure that we're continuing from the block for which we've already fetched L1->L2 message logs, independently if there was a matching transaction on L2?

kkovaacs · 2024-09-24T08:41:15Z

crates/storage/src/schema/revision_0065.rs

+            l1_tx_hash BLOB NOT NULL,
+            l2_tx_hash BLOB NOT NULL
+        );
+        CREATE INDEX idx_l1_handler_txs_l1_tx_hash ON l1_handler_txs(l1_tx_hash);


Another thought: the specification seems to require us returning the L2 transaction statuses for an L1 transaction hash in sending order. We do not currently store any information related to that -- we probably should?

t00ts added this to the JSON-RPC 0.8.0 milestone Sep 19, 2024

t00ts force-pushed the t00ts/messaging branch 5 times, most recently from 3fcee20 to fc4b2eb Compare September 23, 2024 09:08

t00ts marked this pull request as ready for review September 23, 2024 09:09

t00ts requested a review from a team as a code owner September 23, 2024 09:09

sistemd reviewed Sep 23, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

sistemd reviewed Sep 23, 2024

View reviewed changes

crates/common/src/lib.rs Outdated Show resolved Hide resolved

sistemd reviewed Sep 23, 2024

View reviewed changes

crates/ethereum/src/lib.rs Outdated Show resolved Hide resolved

sistemd reviewed Sep 23, 2024

View reviewed changes

crates/ethereum/src/lib.rs Outdated Show resolved Hide resolved

CHr15F0x reviewed Sep 23, 2024

View reviewed changes

crates/ethereum/src/lib.rs Outdated Show resolved Hide resolved

sistemd reviewed Sep 23, 2024

View reviewed changes

crates/storage/src/schema/revision_0065.rs Show resolved Hide resolved

CHr15F0x reviewed Sep 23, 2024

View reviewed changes

crates/pathfinder/src/state/sync.rs Outdated Show resolved Hide resolved

t00ts force-pushed the t00ts/messaging branch from fc4b2eb to 6ff249d Compare September 23, 2024 13:51

t00ts added 9 commits September 23, 2024 15:54

feat: store L1->L2 message logs

e1ee043

feat: introduce get_logs to Ethereum client

a9c7160

feat: bulk sync L1ToL2MessageLoglogs

8824338

chore: migrate db

4bbf4b0

chore: switch useful logs to debug level

d18e6b7

chore: update CHANGELOG.md

bd16e95

chore: remove async_recursion macro

9795b5a

chore: account for mainnet & sepolia for const block numbers

5fbd69f

chore: add explanation on get logs

9382b78

t00ts force-pushed the t00ts/messaging branch from 6ff249d to ab30a2c Compare September 23, 2024 13:55

feat: update Transaction::purge_block to include l1_handler txs a…

a935e35

…nd `L1ToL2MessageLog`s

t00ts force-pushed the t00ts/messaging branch from ab30a2c to a935e35 Compare September 23, 2024 15:35

kkovaacs reviewed Sep 24, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for L1 to L2 message log syncing #2251

Add support for L1 to L2 message log syncing #2251

t00ts commented Sep 19, 2024

kkovaacs left a comment

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

kkovaacs Sep 24, 2024

	tracing::debug!("Got an L1ToL2Message: {:?}", msg.message_hash);
	tracing::trace!(message_hash=%msg.message_hash, "Got an L1ToL2Message");

	tracing::debug!("Found L2 tx for L1 Tx {:?}", l1_tx_hash);
	tracing::trace!(%l1_tx_hash, %l2_tx_hash, "Found L2 tx for L1 tx);

	tracing::debug!("L2 tx NOT found for L1 Tx {:?}", l1_tx_hash);
	tracing::trace!(%l1_tx_hash, "L2 tx NOT found for L1 tx");

	row.get::<_, Option<u64>>(0)?,
	row.get_optional_block_number(0)?,

	tracing::trace!("Removed L1 to L2 message log: {:?}", message_hash);
	tracing::trace!(?message_hash, "Removed L1 to L2 message log);

Add support for L1 to L2 message log syncing #2251

Are you sure you want to change the base?

Add support for L1 to L2 message log syncing #2251

Conversation

t00ts commented Sep 19, 2024

Key Changes

kkovaacs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment