Implement a debug mode to simplify quick bug analysis

[roll]

Overview

We need a debug mode available because errors are caught in a few places in the framework. For example, the CLI catches all the errors or Report.from_validate.

We might use an env var or a flag

[fjuniorr]

@roll just to add additional motivation for this I've had a hard time tracking down a wrong delimiter (,) specified in the csv dialect of a resource when the actual csv file was using ;.

Here is an example of the message:

from frictionless import Package, validate_resource
from pprint import pprint

dp = Package('https://raw.githubusercontent.com/dados-mg/datapackage-reprex/bad-dialect-delimiter/datapackage.json')

report = validate_resource(dp.get_resource('pib-per-capita'))

pprint(report)

{'errors': [{'code': 'task-error',
             'description': 'General task-level error.',
             'message': "The task has an error: 'fieldPosition'",
             'name': 'Task Error',
             'note': "'fieldPosition'",
             'tags': []}],
 'stats': {'errors': 1, 'tasks': 0},
 'tasks': [],
 'time': 2.439,
 'valid': False,
 'version': '4.16.6'}

I think that this more obscure message has something to do with the fact that the resource pib-per-capita has a foreignKeys constraint, because the error in the other resource (ie. validate_resource(dp.get_resource('estados'))) is more clear.

Ps. I've found #903 in the process of investigating this problem and initially though they could be related. 😅

[aivuk]

@roll What do you think a useful debug mode would be, just add some logging.debug where we currently have exceptions or are you thinking in something else?

[roll]

For a starter,

$ frictionless validate table.csv --debug

that won't be swallowing exceptions as a part of validation report.

In general, it's about internal error reporting -- we don't need any interactive debugging etc ATM

[aivuk]

Ok, if we don't swallow the exception, what we are going to do with it? I did some tests raising the exception on the CLI when a validation report was being created, but I don't see which additional information to add. Show the exception and which line it happened?

[roll]

I think just letting the app fail with the exception would be a good start.

Another question - how to keep debug/not-debug state. I think we need to create a system.debug flag which can be set by users and in our case by CLI

[roll]

Hi @aivuk,

Let's park this issue for now (moving to Blocked) as I'm reworking exception for v5 so we might not need to the way we discussed

[roll]

DONE in #1119