Eip7594

[n8wb]

No description provided.

[holgerd77]

Ah ok, pretty cool, didn't see this yet, so for reference: this solves #19

[holgerd77]

First review round!

[holgerd77]

Hi hi, can you also drop 2-3 sentences what brought you here? Also if you plan to complete this to be be integrated here in the library? That would be helpful for some context!

[n8wb]

Hi hi, can you also drop 2-3 sentences what brought you here? Also if you plan to complete this to be be integrated here in the library? That would be helpful for some context!

Hello, I am Nate, and I am working on Blobkit as a project of the ETHCF. We use the kzg-wasm library in our implementation for portability and browser compatibility. The upcoming Fukasa fork on mainnet this year (and currently live on sepolia) implements EIP7594, which breaks the current logic. Since we had to make this change on our end to maintain blob functionality, I figured I would pass along the change as well.

[n8wb]

I notice that CI failed on npm run test:browser, it passes locally, after looking at the error message, it seems that one of the libraries is not compatible with nodejs 18, is support for this version still needed (it reached EOL in April 2025)?

[holgerd77]

No, Node 18 can go away!

Also, did you give the new version a test in the context of our tx library here with the 4844 txs? So are these APIs aligned already that this work to pass this in like in the example with:

const common = new Common({
    chain: Mainnet,
    hardfork: Hardfork.Cancun,
    customCrypto: { kzg },
  })

[n8wb]

No, Node 18 can go away!

Also, did you give the new version a test in the context of our tx library here with the 4844 txs? So are these APIs aligned already that this work to pass this in like in the example with:

const common = new Common({
    chain: Mainnet,
    hardfork: Hardfork.Cancun,
    customCrypto: { kzg },
  })

No, I haven't, I will give it a try, thanks!

PS I have a couple more tests and one more function coming soon, so I switched to draft until that is fully ready.

[n8wb]

Ok, it works with that example now.

package.json

{
  "name": "test",
  "version": "1.0.0",
  "description": "",
  "license": "ISC",
  "author": "",
  "type": "commonjs",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "@blobkit/kzg-wasm": "2.2.17",
    "@ethereumjs/common": "^10.0.0",
    "@ethereumjs/tx": "^10.0.0",
    "@ethereumjs/util": "^10.0.0",
    "kzg-wasm": "^0.5.0"
  }
}

index.js

const { Common, Hardfork, Mainnet } = require('@ethereumjs/common')
const { createBlob4844Tx } = require('@ethereumjs/tx')
const { bytesToHex } = require('@ethereumjs/util')
const { loadKZG } = require('@blobkit/kzg-wasm')

const main = async () => {
  const kzg = await loadKZG()
  kzg.loadTrustedSetup()
  const common = new Common({
    chain: Mainnet,
    hardfork: Hardfork.Cancun,
    customCrypto: { kzg },
  })

  const txData = {
    data: '0x1a8451e600000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
    gasLimit: '0x02625a00',
    maxPriorityFeePerGas: '0x01',
    maxFeePerGas: '0xff',
    maxFeePerBlobGas: '0xfff',
    nonce: '0x00',
    to: '0xcccccccccccccccccccccccccccccccccccccccc',
    value: '0x0186a0',
    v: '0x01',
    r: '0xafb6e247b1c490e284053c87ab5f6b59e219d51f743f7a4d83e400782bc7e4b9',
    s: '0x479a268e0e0acd4de3f1e28e4fac2a6b32a4195e8dfa9d19147abe8807aa6f64',
    chainId: '0x01',
    accessList: [],
    type: '0x05',
    blobsData: ['abcd'],
  }

  const tx = createBlob4844Tx(txData, { common })

  console.log(bytesToHex(tx.hash())) //0x3c3e7c5e09c250d2200bcc3530f4a9088d7e3fb4ea3f4fccfd09f535a3539e84

  // To send a transaction via RPC, you can something like this:
  // const rawTx = tx.sign(privateKeyBytes).serializeNetworkWrapper()
  // myRPCClient.request('eth_sendRawTransaction', [rawTx]) // submits a transaction via RPC
}

void main()

[holgerd77]

Ok, published https://www.npmjs.com/package/kzg-wasm/v/1.0.0-rc.1, can you retry?

[n8wb]

It works, I did another test tx https://sepolia.etherscan.io/tx/0xdec15e5b453f00e5a632dc7635311baf813da3ac30d5629f0f0f7521c68edea7

[holgerd77]

LGTM

[holgerd77]

Ok, I'll take this in now, won't imminently release though, rather sometime between Monday and Wednesday. Will continue to check the API a bit, eventually add some additional docs, generally need some time to work myself in here!

[holgerd77]

One basic question still, so the old (pre-7594) and the new post-7594 blob handling/proof generation should be working with this, right?

[holgerd77]

#21

Let me know if you have any remarks (some API docs in README).

[n8wb]

One basic question still, so the old (pre-7594) and the new post-7594 blob handling/proof generation should be working with this, right?

Yes, this compatibility is very important since not all networks support 7594 yet. The only breaking change is in the trusted setup, where we now take in 3 parameters g1_monomial, g2_monomial, and g1_lagrange, when before it was g1, n1, g2, n2. This matches the C implementation which takes in those 3.

[n8wb]

Unless you are providing a custom trusted setup, you should be able to safely upgrade to this version.

[n8wb]

Side note to anyone who finds this while debugging EIP7594 transaction's not submitting: They are serialized slightly differently from EIP4844, example, notice wrapperVersion

[holgerd77]

Hey hey, we have some KZG API tests in the Util package here, so we have defined a KZG interface in Util as well to ensure API compatibility with both the JS version and the WASM version.

Guess your already worked in this direction, but just to recap: so all the 4844 methods there should remain "untouched" respectively be kept/exposed in a way that these are still available without breaking changes. For the new cell-based methods from 7594 we are still somewhat flexible - nothing released yet - and we can still adjust names or parameter input.

That's just as some general information, I haven't looked deeper yet. At the moment we are testing the kzg-wasm library in the Util package a bit, I think I want to go a bit deeper here and also re-integrate the WASM version in tests in minimally the Tx package and maybe other relevant packages before we release, so that we can ensure that things work seamleassly together.

As a concrete starter: I've opened this PR ethereumjs/ethereumjs-monorepo#4153 for Util, where I started with adjusting the Util KZG tests. The test calling into the computeBlobProof() method is now failing, can you maybe have a look?

This c000.... result looks generally fishy, no matter what trusted setup is used there (I guess), so there seems to be something fundamentally still wrong.

[n8wb]

Hmmmmm, did trusted setup get called in that test? Yeah, I am happy to take a look, let me see

[n8wb]

Ok, I found the issue! The only version would init with only calling loadKZG(), current expects it to look like this

wasm = await loadKZG()
wasm.loadTrustedSetup();

@holgerd77 Adding the latter line to the tests will cause them to pass, I can make a quick PR to make it match the earlier behavior

[n8wb]

#22

[holgerd77]

Thanks a lot, I am also currently looking a bit into naming, this is a chaotic to get through and I need to recreate a bit myself: so, I think with our KZG interface we oriented along the lines of the micro-eth-signer library, just did some checks, it seems, all methods we expose in the interface are available there as well and I think this also generally makes most sense if kzg-wasm has all these methods available at least as aliases, so that one can exchange JS <-> WASM with the least amount of problems.

Let me have a look what there and what's missing:

• blobToKzgCommitment(), yes, alias
• computeBlobProof(), yes, alias
• verifyProof(), honestly not fully sure, is this the same as verifyKZGProof?
• verifyBlobProofBatch(), needs alias from verifyBlobKZGProofBatch() I guess?
• computeCells(), if not available need to be made present by wrapper (?)
• computeCellsAndProofs(), guess needs alias from computeCellsAndKZGProofs()?
• recoverCellsAndProofs(), not fully sure, not fully matches API
• verifyCellKzgProofBatch(), also not fully sure

Hmm. Phew. That's several things to still settle. Can you maybe already address the things which are undisputable in your #22 PR, respectively generally have a first look here?

[n8wb]

Yeah, that should be fairly straightforward

[n8wb]

Added the aliases to the PR

[holgerd77]

Very cool, thanks! Have merged and released rc.2 and updated https://github.com/ethereumjs/ethereumjs-monorepo/pull/4153/files, now tests are timing out:

Not sure, I will also take a look, but maybe you can also directly spot something. Don't want to directly increase the timeouts, since tests passed before in the timeframe and I think it's more likely that something is wrong.

[n8wb]

Very cool, thanks! Have merged and released rc.2 and updated https://github.com/ethereumjs/ethereumjs-monorepo/pull/4153/files, now tests are timing out:

Not sure, I will also take a look, but maybe you can also directly spot something. Don't want to directly increase the timeouts, since tests passed before in the timeframe and I think it's more likely that something is wrong.

Ah, this is due to the precompute generation taking too long, set the timeout (only for beforeAll) to 30,000, and it should be solved. (Also could set precompute to 0)

[n8wb]

Precompute generation happens only once and it speeds up subsequent calculations, 8 is usually the sweet spot in production, but in testing, it slows it down

[holgerd77]

Cool, ok, works https://github.com/ethereumjs/ethereumjs-monorepo/pull/4153/files, aliasing also seems ok, could fully replace all the wrapping we - also - still had internally with the new version, and tests pass! 🎉

That's some great progress, need to stop for today, will continute tomorrow a bit and add the cell methods to these simple API tests and also integrate with the tx library, and then we can hopefully release one or two days later!

[holgerd77]

Hey there, I have now integrated the new kzg-wasm version to run with our tx tests here ethereumjs/ethereumjs-monorepo#4153 and things seem to work, so that's great! I would still want to have a closer look if/where KZG functionality is actually applied and if our tests are sufficient.

Two small API things, I have started to address in #23:

1. I would want to switch the order of the precompute and trustedSetup init arguments, that seems a lot more user friendly to me that people do not need to re-create the mainnet setup when most people will just want to change the precompute (actually I personally am not able at all - so to re-create the mainnet setup, likely also since I did not want to put too much time ressources into figuring out the format, maybe you can just provide a test case here (if easy) which does this, so to manually load a working trusted setup (likely mainnet for pragmatic reasons). Not sure if this needs the trusted setup from https://github.com/paulmillr/trusted-setups as dev dependency - maybe that's generally a good thing to have (?)).

2. After sleeping a night over it I thought that it is indeed a lot more user friendly to have (near) instant object instantiation (when calling loadKZG() with the trusted setup) by default, since otherwise people will be constantly irritated that this goes so long and will assume that something is wrong when first-trying simple examples. We would then very prominently point out in the docs how to adjust for production usage.

Would you go along so far?

I thought this would be accomplished by setting precompute to 0. However this does not change the initialization times more or less (still 15+ seconds on my machine). Does this rather point to something being wrong, or am I using precompute in a wrong way? What additionally irritates me that a) the JS library is acting instantly in base configuration and b) I thought that also this library was pretty quick on setup before the changes.

Would be great if you can still address these points. Thanks!

[n8wb]

Switching the order of arguments makes sense.
I agree that it is much more user friendly, PR #22 made it load immediately when calling loadKZG.

Yeah, the loading takes a while, I do remember it taking a while with the previous version when we were using it, I won't have the time to take an in depth look until later this week, however, here is a small preliminary.
The hex to bytes conversion takes 14ms on my machine.

Here is the C function I added, I don't think that is the cause because it really doesn't do anything besides call the load_trusted_setup function providing the lengths of the arguments

uint32_t load_trusted_setup_wasm(
    uint8_t* g1_monomial_bytes,
    uint8_t* g1_lagrange_bytes,
    uint8_t* g2_monomial_bytes,
    uint64_t precompute) 
{

    uint32_t ret = load_trusted_setup(
        s,
        g1_monomial_bytes,
        NUM_G1_POINTS * BYTES_PER_G1,
        g1_lagrange_bytes,
        NUM_G1_POINTS * BYTES_PER_G1,
        g2_monomial_bytes,
        NUM_G2_POINTS * BYTES_PER_G2,
        precompute
    );
    if (ret == C_KZG_OK) {
       return 0;
    }
    return ret | ((get_last_setting_error()&0xFFFF) << 16);
}

When running the setup in native C, it takes 2 seconds to load the trusted setup, so WASM setup would be expected to be 4-5 seconds.
Maybe nodejs <> WASM interface is doing something weird?

[holgerd77]

Ok, thanks for the additional look, I will then take things in with #23 and release that as the final version, also do not have the time to investigate further and I guess for now it's ok (also still possible to do further non-breaking improvements/releases down the line)!

Thanks so far for all the contributions and the ongoing persistence in "making this round"! Very much appreciate that! 👍 💯 ❤️ 😄

[holgerd77]

Ah, this is still a somewhat bigger one: so this usage of the Node.js url primitive breaks direct browser compatibility. I am not fully sure about the former setup, but I think we made the kzg.js to be able to directly import the WASM or something? Not fully sure, but we should change that. We did a lot to get all Node.js primitives out of our code base, since this makes it a lot easier to use stuff in the browser.

[holgerd77]

So this leads to errors like:

I know this can be polyfilled or something, but that's exactly what we would want to avoid.

No rush though, totally ok if this goes into next week before release so that we can take the time to settle the last things.

[n8wb]

Good point, I can look into this more as well, the reason I added the loaders was due to wasm path resolution issues, need to add something to make it browser compatible as well. When we were using, I think it was v0.4.0, in blobkit, I had to copy and paste the wasm file to wasm/kzg.wasm.

[holgerd77]

Ok, I have added some easier way to test if the distribution build correctly (new npm scripts "test:src" and "test:dist" as well as some developer docs on the script/build specifics (AI supported) here #24. Hope that helps!

Regarding the path resolution issues there are some dedicated package.json scripts fixWasmDirInNode and fixWasmDirInWeb, so maybe these are related to this specific problem.

[holgerd77]

Hey hey, do you think you could have a look into this (the url dependency thing) during the week? I know it's sometimes hard to stick for some "longer than expected time" on these kind of contributions on the side of the "normal work", but this would I think be really the last one and it would be really valuable to also have this in for the releases, then this is "round" on all sides. 🙂

[n8wb]

Browser support is a bit outside my expertise, but it looks like the default WASM binary loader works for the browser version, since those tests pass (The older versions always used the default loader).

[holgerd77]

Ok, let's give it a try. Will release and run our browser tests.

[holgerd77]

Ok, we'll see how it goes ethereumjs/ethereumjs-monorepo#4169, test-all-browser is the relevant CI run.

[holgerd77]

Hmm, not, not really yet. Do not have time to have a deeper look right now, will see if we can come up with a somewhat more solid direct testing on the kzg-wasm side, these round-trips are unnecessary and costly.

[n8wb]

Yeah, it rather costly... I think I found a solution to that issue, though. #26 whenever you get a chance.
It looks like having locateFile defined breaks the browser version; when I made it undefined, it worked. The version in that PR worked when I injected it and then ran the tests.