Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the test pass on windows #1478

Merged

Conversation

aucampia
Copy link
Member

@aucampia aucampia commented Nov 24, 2021

In rdflib:

  • Fix rdflib.plugins.shared.jsonld.util.norm_url on Windows
    On windows normpath and sep is not appropriate for URLs, use posixpath
    so that the behaviour is consistent.

  • Fix handling of file URIs in Graph.serialize on windows
    Should use url2pathname(path) to get a path from the path segment of
    URLs

In tests:

  • Fix handling of file created by mkstemp
    Should first call os.close on the OS-level handle before removing the
    file.

  • Fix how file URIs are created so they work properly on windows.

  • Skip tests which are sensitive to unicode normalization and carriage
    return as these do not pass on windows (about 5 tests).

I would recommend merging #1461 after this so that we don't have regressions on Windows or MacOS.

@aucampia
Copy link
Member Author

iwana@DESKTOP-KLQG489 MINGW64 ~/d/github.com/iafork/rdflib (iwana-20211116T1039-fix_windows)
$ uname -a
MINGW64_NT-10.0-19042 DESKTOP-KLQG489 3.1.7-340.x86_64 2021-10-12 16:29 UTC x86_64 Msys

iwana@DESKTOP-KLQG489 MINGW64 ~/d/github.com/iafork/rdflib (iwana-20211116T1039-fix_windows)
$ git log -1
commit 65588b008044e93410f8a744e0b07790f4727ded (HEAD -> iwana-20211116T1039-fix_windows, origin/iwana-20211116T1039-fix_windows)
Author: Iwan Aucamp <aucampia@gmail.com>
Date:   Wed Nov 24 22:56:20 2021 +0100

    Make the test pass on windows

    In rdflib:

    - Fix rdflib.plugins.shared.jsonld.util.norm_url on Windows
      On windows normpath and sep is not appropriate for URLs, use posixpath
      so that the behaviour is consistent.

    - Fix handling of file URIs in Graph.serialize on windows
      Should use url2pathname(path) to get a path from the path segment of
      URLs

    In tests:

    - Fix handling of file created by mkstemp
      Should first call os.close on the OS-level handle before removing the
      file.

    - Fix how file URIs are created so they work properly on windows.

    - Skip tests which are sensitive to unicode normalization and carriage
      return as these do not pass on windows (about 5 tests).

iwana@DESKTOP-KLQG489 MINGW64 ~/d/github.com/iafork/rdflib (iwana-20211116T1039-fix_windows)
$ .tox/py39/Scripts/pytest "test"
=========================================================================== test session starts ============================================================================
platform win32 -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: C:\Users\iwana\d\github.com\iafork\rdflib, configfile: setup.cfg
plugins: cov-3.0.0, subtests-0.5.0
collected 3936 items

test\helper.py .                                                                                                                                                      [  0%]
test\test_aggregate_graphs.py .                                                                                                                                       [  0%] 
test\test_auditable.py ........................                                                                                                                       [  0%]
test\test_b64_binary.py .                                                                                                                                             [  0%] 
test\test_batch_add.py ..........                                                                                                                                     [  0%]
test\test_bnode_ncname.py .                                                                                                                                           [  0%] 
test\test_canonicalization.py ..........x                                                                                                                             [  1%]
test\test_comparison.py ..........                                                                                                                                    [  1%] 
test\test_conjunctive_graph.py ....                                                                                                                                   [  1%] 
test\test_container.py .................                                                                                                                              [  2%]
test\test_conventions.py .                                                                                                                                            [  2%] 
test\test_create_input_source.py ..                                                                                                                                   [  2%] 
test\test_csv2rdf.py ..                                                                                                                                               [  2%]
test\test_dataset.py ssss....                                                                                                                                         [  2%]
test\test_datetime.py .......                                                                                                                                         [  2%] 
test\test_dawg.py .........................................ss........................................................................................................ [  6%]
..................................................................................................................................................................... [ 10%]
.......................................................s...s....ss..ssssssssssss....ssss...........ss..sss.sss..sssssssssssssss...................................... [ 14%]
......................................................................................sssssss........................................................................ [ 18%]
...........................................................................................................................                                           [ 21%]
test\test_diff.py ..x                                                                                                                                                 [ 22%]
test\test_duration.py ......                                                                                                                                          [ 22%]
test\test_empty_xml_base.py ..                                                                                                                                        [ 22%] 
test\test_evaluate_bind.py ...                                                                                                                                        [ 22%]
test\test_events.py .                                                                                                                                                 [ 22%] 
test\test_expressions.py ......                                                                                                                                       [ 22%]
test\test_extras_external_graph_libs.py ss                                                                                                                            [ 22%] 
test\test_finalnewline.py .                                                                                                                                           [ 22%] 
test\test_graph.py ssssssssss..........                                                                                                                               [ 23%]
test\test_graph_context.py ssssssssss..........                                                                                                                       [ 23%]
test\test_graph_formula.py ..s                                                                                                                                        [ 23%] 
test\test_graph_http.py ....                                                                                                                                          [ 23%]
test\test_graph_items.py .                                                                                                                                            [ 23%] 
test\test_graph_operator.py ...                                                                                                                                       [ 23%] 
test\test_hex_binary.py ..                                                                                                                                            [ 23%] 
test\test_initbindings.py ..........................                                                                                                                  [ 24%]
test\test_issue084.py ........                                                                                                                                        [ 24%] 
test\test_issue1043.py .                                                                                                                                              [ 24%]
test\test_issue1141.py ...                                                                                                                                            [ 24%] 
test\test_issue1160.py .                                                                                                                                              [ 24%]
test\test_issue160.py .                                                                                                                                               [ 24%] 
test\test_issue161.py .                                                                                                                                               [ 24%] 
test\test_issue184.py .                                                                                                                                               [ 25%]
test\test_issue190.py xx                                                                                                                                              [ 25%] 
test\test_issue200.py s                                                                                                                                               [ 25%] 
test\test_issue209.py .                                                                                                                                               [ 25%]
test\test_issue223.py .                                                                                                                                               [ 25%] 
test\test_issue247.py ..                                                                                                                                              [ 25%] 
test\test_issue248.py .                                                                                                                                               [ 25%]
test\test_issue274.py .................................                                                                                                               [ 26%]
test\test_issue363.py ..                                                                                                                                              [ 26%] 
test\test_issue379.py ..                                                                                                                                              [ 26%]
test\test_issue381.py .......                                                                                                                                         [ 26%]
test\test_issue432.py .                                                                                                                                               [ 26%] 
test\test_issue446.py .                                                                                                                                               [ 26%]
test\test_issue492.py .                                                                                                                                               [ 26%] 
test\test_issue523.py .                                                                                                                                               [ 26%] 
test\test_issue532.py .                                                                                                                                               [ 26%]
test\test_issue535.py .                                                                                                                                               [ 26%] 
test\test_issue545.py .                                                                                                                                               [ 26%] 
test\test_issue554.py .                                                                                                                                               [ 26%] 
test\test_issue563.py ..                                                                                                                                              [ 26%]
test\test_issue579.py .                                                                                                                                               [ 26%] 
test\test_issue604.py .                                                                                                                                               [ 26%] 
test\test_issue655.py .                                                                                                                                               [ 26%] 
test\test_issue715.py .                                                                                                                                               [ 26%] 
test\test_issue733.py ..                                                                                                                                              [ 26%]
test\test_issue801.py .                                                                                                                                               [ 26%] 
test\test_issue910.py ....                                                                                                                                            [ 26%]
test\test_issue920.py .                                                                                                                                               [ 26%] 
test\test_issue923.py .                                                                                                                                               [ 26%] 
test\test_issue953.py .                                                                                                                                               [ 26%] 
test\test_issue977.py ..                                                                                                                                              [ 26%]
test\test_issue_git_200.py .                                                                                                                                          [ 27%] 
test\test_issue_git_336.py .                                                                                                                                          [ 27%] 
test\test_literal.py ..................x                                                                                                                              [ 27%]
test\test_memory_store.py ..                                                                                                                                          [ 27%] 
test\test_mulpath_n3.py .                                                                                                                                             [ 27%] 
test\test_n3.py ................                                                                                                                                      [ 27%]
test\test_n3_suite.py ............................................................                                                                                    [ 29%]
test\test_namespace.py ........x..........                                                                                                                            [ 30%]
test\test_nodepickler.py ...                                                                                                                                          [ 30%] 
test\test_normalizedString.py ....                                                                                                                                    [ 30%] 
test\test_nquads.py ...........                                                                                                                                       [ 30%]
test\test_nquads_w3c.py .....................................................................................                                                         [ 32%]
test\test_nt_misc.py ......................                                                                                                                           [ 33%]
test\test_nt_w3c.py ................................................                                                                                                  [ 34%]
test\test_parse_file_guess_format.py ....                                                                                                                             [ 34%]
test\test_parser.py ....                                                                                                                                              [ 34%] 
test\test_parser_helpers.py .                                                                                                                                         [ 34%]
test\test_parser_reads_from_pathlike_object.py .                                                                                                                      [ 34%] 
test\test_path_div_future.py .                                                                                                                                        [ 34%] 
test\test_prefixTypes.py .                                                                                                                                            [ 34%] 
test\test_preflabel.py .....                                                                                                                                          [ 34%] 
test\test_prettyxml.py .......                                                                                                                                        [ 35%]
test\test_rdf_lists.py ..                                                                                                                                             [ 35%] 
test\test_rdfxml.py ..                                                                                                                                                [ 35%]
test\test_roundtrip.py .............................................................................................................................................. [ 38%]
..................................................................................................................................................................... [ 42%]
..................................................................................................................................................................... [ 47%]
..................................................................................................................................................................... [ 51%]
..................................................................................................................................................................... [ 55%]
..................................................................................................................................................................... [ 59%]
..................................................................................................................................................................... [ 63%]
....................................................................................................................................................................  [ 68%]
test\test_seq.py ..                                                                                                                                                   [ 68%]
test\test_serialize.py ....                                                                                                                                           [ 68%]
test\test_serializexml.py .....                                                                                                                                       [ 68%]
test\test_slice.py .                                                                                                                                                  [ 68%] 
test\test_sparql.py .........                                                                                                                                         [ 68%]
test\test_sparql_agg_distinct.py ....                                                                                                                                 [ 68%]
test\test_sparql_agg_undef.py .......                                                                                                                                 [ 68%]
test\test_sparql_construct_bindings.py .                                                                                                                              [ 68%]
test\test_sparql_datetime.py ....                                                                                                                                     [ 68%]
test\test_sparql_operators.py ...                                                                                                                                     [ 69%] 
test\test_sparql_parser.py ..                                                                                                                                         [ 69%]
test\test_sparql_service.py .......                                                                                                                                   [ 69%]
test\test_sparqlstore.py ........                                                                                                                                     [ 69%]
test\test_sparqlupdatestore.py ssssssssssss                                                                                                                           [ 69%] 
test\test_sparqlupdatestore_mock.py .                                                                                                                                 [ 69%]
test\test_store.py ..                                                                                                                                                 [ 69%] 
test\test_store_berkeleydb.py ssssss                                                                                                                                  [ 70%] 
test\test_swap_n3.py .....sss.s..ss..s....ssss.s.s.                                                                                                                   [ 70%]
test\test_term.py .......                                                                                                                                             [ 70%]
test\test_testutils.py .................                                                                                                                              [ 71%]
test\test_tokendatatype.py .....                                                                                                                                      [ 71%] 
test\test_trig.py .........s.                                                                                                                                         [ 71%]
test\test_trig_w3c.py ..............................................................................x................................................................ [ 75%]
..............................................................................................................................xx..................................... [ 79%]
...........................                                                                                                                                           [ 80%]
test\test_trix_parse.py ...                                                                                                                                           [ 80%]
test\test_trix_serialize.py ..                                                                                                                                        [ 80%] 
test\test_tsvresults.py .                                                                                                                                             [ 80%] 
test\test_turtle2.py .                                                                                                                                                [ 80%] 
test\test_turtle_serialize.py .....                                                                                                                                   [ 80%]
test\test_turtle_sort_issue613.py ..                                                                                                                                  [ 80%] 
test\test_turtle_w3c.py ...................................................................x......................................................................... [ 84%]
....................................................................................................................xx............................                    [ 87%]
test\test_util.py ............................x...............                                                                                                        [ 89%]
test\test_wide_python.py .                                                                                                                                            [ 89%] 
test\test_xmlliterals.py ....                                                                                                                                         [ 89%]
test\testutils.py ..                                                                                                                                                  [ 89%]
test\jsonld\test_api.py .                                                                                                                                             [ 89%] 
test\jsonld\test_compaction.py ...........                                                                                                                            [ 89%]
test\jsonld\test_context.py ................                                                                                                                          [ 89%]
test\jsonld\test_localsuite.py ..                                                                                                                                     [ 90%] 
test\jsonld\test_named_graphs.py ...                                                                                                                                  [ 90%] 
test\jsonld\test_onedotone.py ....................................................................................................................................... [ 93%]
..............................................................................................................................                                        [ 96%]
test\jsonld\test_testsuite.py ...............................................................................................................................         [ 99%]
test\jsonld\test_util.py .x                                                                                                                                           [100%]

============================================================================= warnings summary ============================================================================= 
.tox\py39\lib\site-packages\nose\importer.py:12
  C:\Users\iwana\d\github.com\iafork\rdflib\.tox\py39\lib\site-packages\nose\importer.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib; see the
 module's documentation for alternative uses
    from imp import find_module, load_module, acquire_lock, release_lock

rdflib\graph.py:1269: 62 warnings
test/test_canonicalization.py: 14 warnings
test/test_dawg.py: 902 warnings
test/test_prettyxml.py: 10 warnings
test/test_rdfxml.py: 2 warnings
test/test_serializexml.py: 10 warnings
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\graph.py:1269: DeprecationWarning: graph.load() is deprecated, it will be removed in rdflib 6.0.0. Please use graph.parse(
) instead.
    warn(

test/test_dawg.py: 64 warnings
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:900: DeprecationWarning: NotImplemented should not be used in a boolean context
    return not self.__gt__(other) and not self.eq(other)

test/test_finalnewline.py: 1 warning
test/test_nt_misc.py: 2 warnings
test/test_roundtrip.py: 238 warnings
test/test_sparql.py: 1 warning
test/test_sparql_parser.py: 2 warnings
test/test_xmlliterals.py: 1 warning
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\plugins\serializers\nt.py:28: UserWarning: NTSerializer does not use custom encoding.
    warnings.warn("NTSerializer does not use custom encoding.")

test/test_finalnewline.py::testFinalNewline
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\plugins\serializers\jsonld.py:63: UserWarning: JSON should be encoded as unicode. Given encoding was: latin-1
    warnings.warn(

test/test_finalnewline.py::testFinalNewline
test/test_nquads.py::NQuadsParserTest::test_serialize
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\plugins\serializers\nquads.py:24: UserWarning: NQuadsSerializer does not use custom encoding.
    warnings.warn("NQuadsSerializer does not use custom encoding.")

test/test_literal.py::TestParseBoolean::testNonFalseBoolean
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:1430: UserWarning: Parsing weird boolean, 'abcd' does not map to True or False
    warnings.warn(

test/test_literal.py::TestParseBoolean::testNonFalseBoolean
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:1430: UserWarning: Parsing weird boolean, '10' does not map to True or False
    warnings.warn(

test/test_n3_suite.py: 1 warning
test/test_roundtrip.py: 10 warnings
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:1318: UserWarning: Serializing weird numerical rdflib.term.Literal('xy.z', datatype=rdflib.term.URIRef('http://www
.w3.org/2001/XMLSchema#double'))
    warnings.warn("Serializing weird numerical %r" % self)

test/test_n3_suite.py: 1 warning
test/test_roundtrip.py: 10 warnings
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:1318: UserWarning: Serializing weird numerical rdflib.term.Literal('+1.0z', datatype=rdflib.term.URIRef('http://ww
w.w3.org/2001/XMLSchema#double'))
    warnings.warn("Serializing weird numerical %r" % self)

test/test_n3_suite.py: 1 warning
test/test_roundtrip.py: 10 warnings
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\term.py:1318: UserWarning: Serializing weird numerical rdflib.term.Literal('ab.c', datatype=rdflib.term.URIRef('http://www
.w3.org/2001/XMLSchema#double'))
    warnings.warn("Serializing weird numerical %r" % self)

test/test_namespace.py::NamespacePrefixTest::test_closed_namespace
  C:\Users\iwana\d\github.com\iafork\rdflib\test\test_namespace.py:214: UserWarning: DefinedNamespace does not address deprecated properties
    warn("DefinedNamespace does not address deprecated properties")

test/test_preflabel.py::TestPrefLabel::test_default_label_sorting
test/test_preflabel.py::TestPrefLabel::test_default_preflabel_sorting
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_empty_lang_attr
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_en_lang_attr
test/test_preflabel.py::TestPrefLabel::test_preflabel_lang_sorting_no_lang_attr
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\graph.py:816: DeprecationWarning: graph.preferredLabel() is deprecated and will be removed in rdflib 6.0.0.
    warn(

test/test_rdf_lists.py::OWLCollectionTest::testCollectionRDFXML
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\plugins\serializers\rdfxml.py:262: UserWarning: Assertions on rdflib.term.BNode('Nda02e3894001417f9cac839193502d6b') other
 than RDF.first and RDF.rest are ignored ... including RDF.List
    self.predicate(predicate, object, depth + 1)

test/test_seq.py::SeqTestCase::testSeq
  C:\Users\iwana\d\github.com\iafork\rdflib\rdflib\graph.py:972: DeprecationWarning: graph.seq() is deprecated and will be removed in rdflib 6.0.0.
    warn(

test/test_sparql_datetime.py::test_dateTime_duration_subs
test/test_sparql_datetime.py::test_dateTime_duration_subs
test/test_sparql_datetime.py::test_dateTime_duration_subs
test/test_sparql_datetime.py::test_dateTime_duration_subs
  C:\Users\iwana\d\github.com\iafork\rdflib\.tox\py39\lib\site-packages\isodate\duration.py:267: DeprecationWarning: an integer is required (got type decimal.Decimal).  Impl
icit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
    newdt = other.replace(year=newyear, month=newmonth, day=newday)

test/test_sparql_datetime.py::test_dateTime_duration_add
test/test_sparql_datetime.py::test_dateTime_duration_add
test/test_sparql_datetime.py::test_dateTime_duration_add
test/test_sparql_datetime.py::test_dateTime_duration_add
  C:\Users\iwana\d\github.com\iafork\rdflib\.tox\py39\lib\site-packages\isodate\duration.py:183: DeprecationWarning: an integer is required (got type decimal.Decimal).  Impl
icit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
    newdt = other.replace(year=newyear, month=newmonth, day=newday)

test/test_util.py::TestUtilTermConvert::test_util_from_n3_expectliteralandlangdtype
  C:\Python39\lib\unittest\case.py:550: UserWarning: Code: fr is not defined in namespace XSD
    method()

-- Docs: https://docs.pytest.org/en/stable/warnings.html
====================================================== 3810 passed, 112 skipped, 14 xfailed, 1365 warnings in 48.47s ======================================================= 

In rdflib:

- Fix rdflib.plugins.shared.jsonld.util.norm_url on Windows
  On windows normpath and sep is not appropriate for URLs, use posixpath
  so that the behaviour is consistent.

- Fix handling of file URIs in Graph.serialize on windows
  Should use url2pathname(path) to get a path from the path segment of
  URLs

In tests:

- Fix handling of file created by mkstemp
  Should first call os.close on the OS-level handle before removing the
  file.

- Fix how file URIs are created so they work properly on windows.

- Skip tests which are sensitive to unicode normalization and carriage
  return as these do not pass on windows (about 5 tests).
@aucampia aucampia force-pushed the iwana-20211116T1039-fix_windows branch from 65588b0 to d1fdb1e Compare November 24, 2021 22:18
@nicholascar nicholascar merged commit fb8d007 into RDFLib:master Dec 1, 2021
@aucampia aucampia deleted the iwana-20211116T1039-fix_windows branch December 27, 2021 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants