Skip to content

add Identify errors in plugin#20

Closed
germankay wants to merge 3 commits into
mainfrom
18-indentify-errors
Closed

add Identify errors in plugin#20
germankay wants to merge 3 commits into
mainfrom
18-indentify-errors

Conversation

@germankay

Copy link
Copy Markdown
Contributor

fixes #18

@germankay germankay requested review from avdata99 and Copilot March 27, 2025 12:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds logic to uniquely identify errors in the plugin and prevent duplicate logging and notifications through a new error_logger module.

  • Updates the plugin to call log_error with error deduplication and silencing.
  • Introduces error_logger.py to handle error logging and notification management.
  • Updates the README to document the new error identification and notification features.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
ckanext/push_errors/plugin.py Updates error_handler to use log_error with try/except wrapping
ckanext/push_errors/error_logger.py Adds error_logger module for deduplicated error logging
README.md Adds documentation for the new error identification and management
Comments suppressed due to low confidence (2)

ckanext/push_errors/error_logger.py:11

  • Consider verifying that 'error.traceback' is not None before using it to generate error_id, to avoid always defaulting to 'unknown' when no traceback is available.
def log_error(error, disable_notification=False):

ckanext/push_errors/plugin.py:45

  • [nitpick] The variable name 'log_err' is somewhat terse; consider renaming it to something clearer like 'logging_exception' for improved readability.
except Exception as log_err:

@germankay germankay force-pushed the 18-indentify-errors branch from 2a95dbc to 660cdbd Compare June 2, 2025 13:17
@germankay germankay requested a review from Copilot June 2, 2025 14:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds error identification enhancements to the plugin by deduplicating error logs and enabling notification suppression, as well as corresponding tests and documentation updates.

  • New tests have been added to verify error de-duplication and notification behavior.
  • The plugin now uses log_error with error identification and fallback logging.
  • The error_logger module implements error deduplication using file path and line number.
  • README has been updated to document the new error handling approach.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
ckanext/push_errors/tests/test_error_login.py Adds tests to validate deduplication and optional notification suppression behavior.
ckanext/push_errors/plugin.py Integrates log_error in the error_handler with fallback logging for log_error failures.
ckanext/push_errors/error_logger.py Implements error deduplication by tracking errors based on file path and line number.
README.md Updates documentation for the new error identification and notification management features.

@germankay germankay mentioned this pull request Jun 10, 2025
6 tasks

@avdata99 avdata99 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments but not for change, just to analyze. A new PR is required in the latest comment.

Comment thread README.md
# Some code that raises an exception
raise ValueError("An example error")
except Exception as e:
log_error(e, disable_notification=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disable notification is not expected to do like this.
Disable will be dynamic, this PR is to start uniquely identifying error.
File path + line looks good but disable_notification is not required, see other comments


# Registrar usando log_error (deduplicación y silenciado)
try:
log_error(exception)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will send the push message and after that, we'll sent it again later in this function

Comment on lines +12 to +14
tb = traceback.extract_tb(error.__traceback__)
if tb:
file_path, line_number, _, _ = tb[-1]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It look interesting this way to get the file_path and the line_number ensure it works, document what else we get (the 3º and 4º params) and use it in the new requested PR

file_path, line_number, _, _ = tb[-1]
error_id = f"{file_path}:{line_number}"
else:
error_id = "unknown"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will skip all non identified erros. Always a unique ID is required


log = logging.getLogger(__name__)

logged_errors = set() # To track already logged errors

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be always empty.
For the future PR, we need to use redis or the main CKAN DB to preserve a list of logged errors

@avdata99

avdata99 commented Jun 10, 2025

Copy link
Copy Markdown
Contributor

We have two type of push-error notifications:

  • Server errors captured in make_middleware. This case we have the error object and we are able to traceback
  • Log critical messages: we just have a text from the user. We don't have an error object so we can't traceback. Investigate if we can get the last error in python to use in PushErrorHandler.

So the first goal of this PR is to get a unique ID for all messages before sent them. It could be the filepath+line number for error objects and the whole text for log.critical messages.

You can restart a new PR, this code do not capture the general idea.

Let split this into multiple PRs:
Create a new clean PR that:

  • Creates a helper function called get_error_trace_id which extracts a unique ID from an error object (something like the file path and line number like this PR already uses)
  • Creates a helper function called get_error_log_id which extracts a unique ID from a log records. Use a PushErrorHandler log record as parameter. Investigate is traceback can get the last error in Python. If not, use the message as rthe unique ID. Considering this cuold be long, hash it with some function to reduce it

Use both function and add the unique ID to the push message
Let see how it goes and define the next step

This was referenced Jun 11, 2025
@germankay germankay closed this Jun 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Identify errors

3 participants