Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Microdata to rdf second edition bak #444

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
82be526
dropping support for python 3.2 as it doesn't allow u"" strings
joernhees Nov 24, 2014
f2d9416
six: added six to requirements
joernhees Nov 24, 2014
c243969
six: modify setup.py so we can sequentially change files from 2to3 to…
joernhees Nov 26, 2014
fec1b23
six: transformed and extended py3compat to be used as six wrapper
joernhees Nov 26, 2014
3c2ad90
six: fix tuple assignment in method signature for py3
joernhees Nov 26, 2014
60b769e
six: __future__ headers
joernhees Nov 26, 2014
a05e083
cleanup: import of hashlib.md5
joernhees Nov 26, 2014
9d02c80
six: relative imports for compat and py3compat
joernhees Nov 26, 2014
bf88b73
six: use urljoin, urlquote, urldefrag from six.moves
joernhees Nov 26, 2014
485b9a9
six: use PY2, PY3 from six, __nonzero__/__bool__ compatibility
joernhees Nov 26, 2014
567cb66
six: unicode --> text_type
joernhees Nov 26, 2014
685ddde
six: basestring --> string_types
joernhees Nov 26, 2014
3949337
six: long --> long_type
joernhees Nov 26, 2014
9d0ecff
minor cleanup
joernhees Nov 26, 2014
94cf43d
six: term.py done
joernhees Nov 26, 2014
5bca727
six: __init__.py done
joernhees Nov 26, 2014
0308d63
six: added list of python files which are 2to3 transformed.
joernhees Nov 27, 2014
e97fa63
six: collection.py headers and exception syntax
joernhees Nov 27, 2014
5c621af
six: compare.py: headers and unicode --> text_type
joernhees Nov 27, 2014
110f306
six: events.py: headers, handling of .keys()
joernhees Nov 27, 2014
fad800e
six: plugin.py: headers, handling of iteritems
joernhees Nov 27, 2014
cf98646
Merge branch 'master' into six_2to3
joernhees Nov 27, 2014
c1f1abe
six: py3compat imports BytesIO and StringIO from six
joernhees Nov 28, 2014
64e8241
six: query.py: headers
joernhees Nov 28, 2014
5e88c13
six: query.py: imports fixed
joernhees Nov 28, 2014
0e6f043
six: query.py: unicode --> text_type
joernhees Nov 28, 2014
6a8941e
six: query.py: __nonzero__ --> __bool__ handling
joernhees Nov 28, 2014
d5b7186
six: query.py done
joernhees Nov 28, 2014
4fe1065
six: resource.py: headers, unicode
joernhees Nov 28, 2014
ccf74a9
six: py3compat import cPickle
joernhees Nov 28, 2014
9f6f966
six: store.py: headers, pickle, exceptions, method sig tuple assignments
joernhees Nov 28, 2014
9f98858
six: util.py: headers, StringIO
joernhees Nov 28, 2014
e7945bd
six: py3compat.py: use efficient py2 BytesIO version
joernhees Nov 28, 2014
4045cca
six: graph.py: headers, imports
joernhees Nov 28, 2014
05a0c49
six: graph.py: method sig tuple assignments, iterators
joernhees Nov 28, 2014
00980aa
six: py3compat.py: added six.moves.urllib.request.pathname2url
joernhees Nov 29, 2014
d42f471
six: namespace.py: headers, imports
joernhees Nov 29, 2014
086a535
six: namespace.py: unicode, string_types
joernhees Nov 29, 2014
6d07984
six: namespace.py: xrange --> range
joernhees Nov 29, 2014
454524c
six: namespace.py: removed unnecessary u"" prefix
joernhees Nov 29, 2014
74b6b0e
six: namespace.py done
joernhees Nov 29, 2014
282b831
six: parser.py: headers, imports, unicode, basestring
joernhees Nov 29, 2014
6d20491
six: run_tests.py: cleanup, py3 compatible
joernhees Nov 30, 2014
2d4276a
six: run_tests_py3.sh updated to base on setup.py build & use run_tes…
joernhees Nov 30, 2014
5f48c64
six: extras/describer.py: headers, rel imports
joernhees Dec 1, 2014
3132d3b
six: extras/infixowl.py: headers
joernhees Dec 1, 2014
39db1bf
six: memory.py: headers, method sig tuple assignments
joernhees Dec 2, 2014
4e47f7a
six: py3compat.py: handlers for iteritems, iterkeys, itervalues
joernhees Dec 2, 2014
793bb2d
six: memory.py: handling of iteritems
joernhees Dec 2, 2014
58dbd14
six: py3compat.py: added binary_type and unichr
joernhees Dec 3, 2014
585c17c
six: parsers/notation3.py: hearders; str, unicode, long, unichr, prin…
joernhees Dec 3, 2014
56847c3
six: parsers/nquads.py: headers, exception syntax
joernhees Dec 4, 2014
724a210
six: util.py: cleanup unused StringIO
joernhees Dec 4, 2014
2eb85a3
six: parsers/ntriples.py: headers, unicode, basetring, BytesIO, print…
joernhees Dec 4, 2014
f21e62d
six: parsers/pyMicrodata/__init__.py: headers, StringIO, urlparse, ba…
joernhees Dec 5, 2014
b082c48
Microdata parser: updated the parser to the latest version of the mic…
iherman Dec 8, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ python:
# - 2.5
- 2.6
- 2.7
- 3.2
# - 3.2
- 3.3
- 3.4
# - "pypy"
Expand Down
6 changes: 5 additions & 1 deletion rdflib/collection.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from rdflib.namespace import RDF
from rdflib.term import BNode
from rdflib.term import Literal
Expand Down Expand Up @@ -260,7 +264,7 @@ def test():

try:
del c[500]
except IndexError, i:
except IndexError as i:
pass

c.append(Literal("5"))
Expand Down
7 changes: 6 additions & 1 deletion rdflib/compare.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@
_:cb558f30e21ddfc05ca53108348338ade8
<http://example.org/ns#label> "B" .
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function


# TODO:
Expand All @@ -93,6 +96,8 @@
import md5
md = md5.new

from .py3compat import text_type


class IsomorphicGraph(ConjunctiveGraph):
"""
Expand Down Expand Up @@ -242,7 +247,7 @@ def _md5_hash(t):
if isinstance(i, tuple):
h.update(_md5_hash(i).encode('ascii'))
else:
h.update(unicode(i).encode("utf8"))
h.update(text_type(i).encode("utf8"))
return h.hexdigest()


Expand Down
7 changes: 5 additions & 2 deletions rdflib/events.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

__doc__ = """
Dirt Simple Events

Expand Down Expand Up @@ -42,8 +46,7 @@ def __init__(self, **kw):
self.__dict__.update(kw)

def __repr__(self):
attrs = self.__dict__.keys()
attrs.sort()
attrs = sorted(self.__dict__.keys())
return '<rdflib.events.Event %s>' % ([a for a in attrs],)


Expand Down
10 changes: 6 additions & 4 deletions rdflib/extras/describer.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import with_statement
from rdflib import py3compat
__doc__ = py3compat.format_doctest_out("""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from ..py3compat import format_doctest_out
__doc__ = format_doctest_out("""
A Describer is a stateful utility for creating RDF statements in a
semi-declarative manner. It has methods for creating literal values, rel and
rev resource relations (somewhat resembling RDFa).
Expand Down Expand Up @@ -113,7 +116,6 @@
from rdflib.term import Identifier
from rdflib.term import Literal
from rdflib.term import URIRef
from rdflib.py3compat import format_doctest_out


class Describer(object):
Expand Down
15 changes: 10 additions & 5 deletions rdflib/extras/infixowl.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from rdflib import py3compat
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

__doc__ = py3compat.format_doctest_out("""
from ..py3compat import PY3
from ..py3compat import format_doctest_out

__doc__ = format_doctest_out("""
RDFLib Python binding for OWL Abstract Syntax

see: http://www.w3.org/TR/owl-semantics/syntax.html
Expand Down Expand Up @@ -1366,7 +1371,7 @@ def __iadd__(self, other):


class EnumeratedClass(OWLRDFListProxy, Class):
py3compat.format_doctest_out("""
format_doctest_out("""
Class for owl:oneOf forms:

OWL Abstract Syntax is used
Expand Down Expand Up @@ -1560,7 +1565,7 @@ def changeOperator(self, newOperator):
... except Exception%s: print(e)
The new operator is already being used!

""" % 'as e' if py3compat.PY3 else ', e'
""" % 'as e' if PY3 else ', e'
assert newOperator != self._operator, \
"The new operator is already being used!"
self.graph.remove((self.identifier, self._operator, self._rdfList.uri))
Expand Down Expand Up @@ -1654,7 +1659,7 @@ def __init__(self,
self.graph.add((self.identifier, RDF.type, OWL_NS.Restriction))
self.graph.remove((self.identifier, RDF.type, OWL_NS.Class))

@py3compat.format_doctest_out
@format_doctest_out
def serialize(self, graph):
"""
>>> g1 = Graph()
Expand Down
66 changes: 34 additions & 32 deletions rdflib/graph.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from rdflib.term import Literal # required for doctests
assert Literal # avoid warning
from rdflib.namespace import Namespace # required for doctests
assert Namespace # avoid warning
from rdflib.py3compat import format_doctest_out
from .py3compat import format_doctest_out

__doc__ = format_doctest_out("""\

Expand Down Expand Up @@ -238,16 +242,6 @@

from hashlib import md5

try:
from io import BytesIO
assert BytesIO
except ImportError:
try:
from cStringIO import StringIO as BytesIO
assert BytesIO
except ImportError:
from StringIO import StringIO as BytesIO
assert BytesIO

from rdflib.namespace import RDF, RDFS, SKOS

Expand All @@ -266,13 +260,14 @@
from rdflib.parser import create_input_source
from rdflib.namespace import NamespaceManager
from rdflib.resource import Resource
from rdflib import py3compat
b = py3compat.b

import os
import shutil
import tempfile
from urlparse import urlparse

from .py3compat import BytesIO
from .py3compat import b
from .py3compat import urlparse

__all__ = [
'Graph', 'ConjunctiveGraph', 'QuotedGraph', 'Seq',
Expand Down Expand Up @@ -384,8 +379,9 @@ def close(self, commit_pending_transaction=False):
self.__store.close(
commit_pending_transaction=commit_pending_transaction)

def add(self, (s, p, o)):
def add(self, triple):
"""Add a triple with self as context"""
s, p, o = triple
assert isinstance(s, Node), \
"Subject %s must be an rdflib term" % (s,)
assert isinstance(p, Node), \
Expand All @@ -403,28 +399,29 @@ def addN(self, quads):
and _assertnode(s,p,o)
)

def remove(self, (s, p, o)):
def remove(self, triple):
"""Remove a triple from the graph

If the triple does not provide a context attribute, removes the triple
from all contexts.
"""
self.__store.remove((s, p, o), context=self)
self.__store.remove(triple, context=self)

def triples(self, (s, p, o)):
def triples(self, triple):
"""Generator over the triple store

Returns triples that match the given triple pattern. If triple pattern
does not provide a context, all contexts will be searched.
"""
s, p, o = triple
if isinstance(p, Path):
for _s, _o in p.eval(self, s, o):
yield (_s, p, _o)
else:
for (s, p, o), cg in self.__store.triples((s, p, o), context=self):
yield (s, p, o)

@py3compat.format_doctest_out
@format_doctest_out
def __getitem__(self, item):
"""
A graph can be "sliced" as a shortcut for the triples method
Expand Down Expand Up @@ -645,7 +642,8 @@ def predicate_objects(self, subject=None):
for s, p, o in self.triples((subject, None, None)):
yield p, o

def triples_choices(self, (subject, predicate, object_), context=None):
def triples_choices(self, triple, context=None):
subject, predicate, object_ = triple
for (s, p, o), cg in self.store.triples_choices(
(subject, predicate, object_), context=self):
yield (s, p, o)
Expand Down Expand Up @@ -681,13 +679,13 @@ def value(self, subject=None, predicate=RDF.value, object=None,
values = self.predicates(subject, object)

try:
retval = values.next()
retval = next(values)
except StopIteration:
retval = default
else:
if any is False:
try:
values.next()
next(values)
msg = ("While trying to find a value for (%s, %s, %s) the"
" following multiple values where found:\n" %
(subject, predicate, object))
Expand All @@ -710,7 +708,7 @@ def label(self, subject, default=''):
return default
return self.value(subject, RDFS.label, default=default, any=True)

@py3compat.format_doctest_out
@format_doctest_out
def preferredLabel(self, subject, lang=None, default=None,
labelProperties=(SKOS.prefLabel, RDFS.label)):
"""
Expand Down Expand Up @@ -767,7 +765,7 @@ def preferredLabel(self, subject, lang=None, default=None,
langfilter = lambda l: True

for labelProp in labelProperties:
labels = filter(langfilter, self.objects(subject, labelProp))
labels = list(filter(langfilter, self.objects(subject, labelProp)))
if len(labels) == 0:
continue
else:
Expand Down Expand Up @@ -1382,9 +1380,9 @@ def quads(self, triple_or_quad=None):
for ctx in cg:
yield s, p, o, ctx

def triples_choices(self, (s, p, o), context=None):
def triples_choices(self, triple, context=None):
"""Iterate over all the triples in the entire conjunctive graph"""

s, p, o = triple
if context is None:
if not self.default_union:
context=self.default_context
Expand Down Expand Up @@ -1643,8 +1641,9 @@ class QuotedGraph(Graph):
def __init__(self, store, identifier):
super(QuotedGraph, self).__init__(store, identifier)

def add(self, (s, p, o)):
def add(self, triple):
"""Add a triple with self as context"""
s, p, o = triple
assert isinstance(s, Node), \
"Subject %s must be an rdflib term" % (s,)
assert isinstance(p, Node), \
Expand Down Expand Up @@ -1797,16 +1796,17 @@ def close(self):
for graph in self.graphs:
graph.close()

def add(self, (s, p, o)):
def add(self, triple):
raise ModificationException()

def addN(self, quads):
raise ModificationException()

def remove(self, (s, p, o)):
def remove(self, triple):
raise ModificationException()

def triples(self, (s, p, o)):
def triples(self, triple):
s, p, o = triple
for graph in self.graphs:
if isinstance(p, Path):
for s, o in p.eval(self, s, o):
Expand All @@ -1825,8 +1825,9 @@ def __contains__(self, triple_or_quad):
return True
return False

def quads(self, (s, p, o)):
def quads(self, triple):
"""Iterate over all the quads in the entire aggregate graph"""
s, p, o = triple
for graph in self.graphs:
for s1, p1, o1 in graph.triples((s, p, o)):
yield (s1, p1, o1, graph)
Expand Down Expand Up @@ -1855,7 +1856,8 @@ def __isub__(self, other):

# Conv. methods

def triples_choices(self, (subject, predicate, object_), context=None):
def triples_choices(self, triple, context=None):
subject, predicate, object_ = triple
for graph in self.graphs:
choices = graph.triples_choices((subject, predicate, object_))
for (s, p, o) in choices:
Expand Down
Loading