SAC-30857: Add exclusion for Bulk objects with Suffix#219
Conversation
| bulk_blacklisted_objects = base_blacklisted_objects.union(UNSUPPORTED_BULK_API_SALESFORCE_OBJECTS) | ||
|
|
||
| if object_names: | ||
| object_names = object_names if isinstance(object_names, list) else list(object_names) |
There was a problem hiding this comment.
Any reason to accept multi data typed args? Why can't we pass always list here?
There was a problem hiding this comment.
It's added as a guard rail to check for any different datatype. Was done because sync was using get_blacklisted_object with stream names as string (which is updated).
Updated the code to handle it better.
There was a problem hiding this comment.
Pull request overview
This PR updates Salesforce object discovery/sync behavior to exclude additional Bulk API–unsupported objects, including dynamically named objects identified by suffix patterns (e.g., *Share, *Feed, *History, *EventRelation), and adds/updates tests to cover the new exclusion logic.
Changes:
- Add suffix-based Bulk API object exclusion logic and thread it into discovery/sync blacklist checks.
- Extend/update Bulk API unsupported object lists and blacklist computation behavior.
- Add unit tests and update tap-tester expectations to account for newly excluded streams.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tap_salesforce/salesforce/__init__.py |
Adds Bulk-unsupported suffix constants/helper and updates get_blacklisted_objects() to support suffix-based exclusions. |
tap_salesforce/__init__.py |
Passes discovered object names into blacklist computation during discovery; updates blacklist checks during sync. |
tests/unittests/test_bulk_api_object_exclusions.py |
Adds unit tests for suffix exclusion and updated blacklist behavior for BULK vs REST. |
tests/base.py |
Updates expected stream set logic for BULK to exclude suffix-pattern streams. |
tests/sfbase.py |
Updates tap-tester expected stream set logic for BULK to exclude suffix-pattern streams. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description of change
JIRA: https://qlik-dev.atlassian.net/browse/SAC-30857
QA steps
Risks
Rollback steps
AI generated code
https://internal.qlik.dev/general/ways-of-working/code-reviews/#guidelines-for-ai-generated-code