All contributors are expected to follow the Code of Conduct.
Please file issues and feature requests on the GitHub issue tracker: https://github.com/adbc-drivers/spark/issues
Potential security vulnerabilities should be reported to security@adbc-drivers.org instead. See the Security Policy.
For basic development, the driver can be built and tested like any Go project.
From the go/ subdirectory:
$ go build ./...
$ go test -tags assert -v ./...This will not produce a shared library, however; that requires invoking the
full build script. You will need pixi installed. From
the go/ subdirectory:
$ pixi run makeTo run the validation suite, you will first need to build the shared library. You will also need to set up an Apache Spark instance. This can be done with Docker Compose:
docker compose up spark4-connect-serverFinally, from the go/ subdirectory:
$ pixi run validate --vendor-version spark4_4.0-connectThis will produce a test report, which can be rendered into a documentation page (using MyST Markdown):
$ pixi run gendocs --output generated/Then look at ./generated/spark.md.
Before opening a pull request:
- Review your changes and make sure no stray files, etc. are included.
- Ensure the Apache license header is at the top of all files.
- Check if there is an existing issue. If not, please file one, unless the change is trivial.
- Assign the issue to yourself by commenting just the word
take. - Run the static checks by installing pre-commit,
then running
pre-commit run --all-filesfrom inside the repository. Make sure all your changes are staged/committed (unstaged changes will be ignored).
When writing the pull request description:
-
Ensure the title follows Conventional Commits format. The component should be
goif it affects the Go driver, or it can be omitted for general maintenance (in general: it should be a directory path relative to the repo root, e.g.go/authwould also be valid if that directory existed). Example titles:feat(go): support GEOGRAPHY data typechore: update action versionsfix!(go): return us instead of ms
Ensure that breaking changes are appropriately flagged with a
!as seen in the last example above. -
Make sure the description ends with
Closes #NNN,Fixes #NNN, or similar, so that the issue will be linked to your pull request.