ó
(sPMc @ se d Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z d d l m
Z
d d l m Z d d l
Z
d d l Z y d d l m Z Wn d d l m Z n Xe j d e j ƒ Z d
„ Z d „ Z d „ Z e j d
e j ƒ Z d „ Z d „ Z d „ Z d „ Z d „ Z d „ Z d „ Z d „ Z d „ Z! d „ Z" d d „ Z$ d S( sÌ
Reconstitute an entry document from the output of the Universal Feed Parser.
The main entry point is called 'reconstitute'. Input parameters are:
results: this is the entire hash table return by the UFP
entry: this is the entry in the hash that you want reconstituted
The value returned is an XML DOM. Every effort is made to convert
everything to unicode, and text fields into either plain text or
well formed XHTML.
Todo:
* extension elements
iÿÿÿÿN( t escape( t minidomt Node( t html5parser( t dom( t md5( t news
[- - ]c C s™ | s
d St | t ƒ rH y | j d ƒ } WqH | j d ƒ } qH Xn t j t | ƒ } | j } | j | ƒ } | j | j | ƒ ƒ | j | ƒ | S( sC utility function to create a child element with the specified textNs utf-8s
iso-8859-1(
t
isinstancet strt decodet illegal_xml_charst subt
invalidatet
ownerDocumentt
createElementt appendChildt createTextNode( t parentt namet valuet xdoct xelement( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt createTextElement s
c C s) d d t t | j d ƒ ƒ ƒ d d S( s replace invalid characters u �t 000i i iüÿÿÿ( t hext ordt group( t c( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyR - s c C sP | j d ƒ } | j d ƒ r: t t | d d ƒ ƒ } n t t | ƒ ƒ } | S( s4 convert numeric character references to characters i t xi ( R t
startswitht unichrt int( R ( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt ncr2c2 s
s \W+c C s] y. t j d | j d ƒ ƒ j ƒ j d ƒ } Wn t j d | ƒ j ƒ } n X| j d ƒ S( s generate a css id from a name t -s utf-8( t nonalphaR R t lowert encodet strip( R ( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt cssid< s
. c C sP | j d ƒ rF | j rF | j } t | d ƒ r3| j ƒ d } q3ní | j d ƒ rj | j rj | j } nÉ | j d ƒ r¨ | j r¨ | j j d t | j ƒ j ƒ } n‹ | j d ƒ ræ | j
ræ | j j d t | j
ƒ j ƒ } nM | j d ƒ r/| j r/| j d j d t | j d j
ƒ j ƒ } n d S| rLt | d | ƒ n | S(
s% copy or compute an id for the entry t idt valuesi t linkt titlet /t summaryt contentN( t has_keyR' t hasattrR( R) R* t title_detailt baseR t hexdigestR, t summary_detailR- R R ( t xentryt entryt entry_id( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyR' D s&
c C sn | j d ƒ sP g | d <| j d ƒ rP | d j i d d 6| j d 6ƒ qP n | j } x| d D]} d | j ƒ k r‚ qd n | j d ƒ } | j d | j d ƒ ƒ | j d ƒ rÕ | j d | j d ƒ ƒ n | j d ƒ r| j d | j d d ƒ ƒ n | j d ƒ r.| j d | j d ƒ ƒ n | j d ƒ rY| j d | j d ƒ ƒ n | j | ƒ qd Wd S(
s copy links to the entry t linksR) t alternatet relt hreft typeR* t lengthN(
R. t appendR) R
t keysR t setAttributet gett NoneR ( R4 R5 R R) t xlink( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyR7 \ s&
( c C sc | s
d St j d | ƒ } t | | | ƒ } t j t j ƒ | ƒ } | j d | j d ƒ ƒ d S( s/ insert a date-formated element into the entry Ns %Y-%m-%dT%H:%M:%SZs
planet:formats utf-8( t timet strftimeR t configt date_formatR? R ( R4 R t parsedt formattedt xdate( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt dateq s c C s | j j d ƒ } | j d ƒ s, | j r0 d S| j d | j d ƒ ƒ | j d ƒ r} | j r} | j d | j d ƒ ƒ n | j d ƒ r± | j r± | j d | j d ƒ ƒ n | j | ƒ d S( Nt categoryt termt schemet label( R
R R. RL R? R@ RM RN R ( R4 t tagt xtag( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyRK y s c C s¯ | s
d S| j } | j | ƒ } | j d d ƒ rP t | d | j d ƒ ƒ n | j | j d ƒ ƒ t | d | j d d ƒ ƒ t | d | j d d ƒ ƒ | j | ƒ d S( s. insert an author-like element into the entry NR t emailt uriR: ( R
R R@ RA R R ( R4 R t detailR t xauthor( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt authorƒ s c C sö | s | j r d Sd } d } | j } | j | ƒ } t | j t ƒ rc | j j d ƒ | _ n | j d ƒ s‘ | j j ƒ j
d ƒ d k r± t | j ƒ | d %ss utf-8R; t htmli R s text/htmlt xhtmli t treet encodingt bodyt divt languages xml:lang(( R RA R
R R t unicodeR$ R. R; R# t findR R t parseStringt documentElementR? R t
HTMLParserR t TreeBuildert parset
childNodest nodeTypeR t ELEMENT_NODEt nodeNamet normalizet lent
firstChildt TEXT_NODER
t searcht dataR R R R R R@ R\ ( R4 R RS t bozoRm t xdivR t xcontentt parserRV RZ R[ ( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyR- “ sb .
* c C s„ | s | r d St | d d d | ƒ } | j d
d ƒ t | d d d | ƒ } | j d d ƒ | j | ƒ | j | ƒ d S(
s$ insert geo location into the entry Ns %s:%st geot lats %fs xmlns:%ss( http://www.w3.org/2003/01/geo/wgs84_pos#t long( s geos lats xmlns:geo( s geos longs xmlns:geo( R R? R ( R4 Rt Rs t xlatt xlong( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt locationÍ s
c C sÕ | j } t | d | j d | j d d ƒ ƒ ƒ t | d | j d d ƒ ƒ t | d | j d d ƒ ƒ | j d ƒ rª | j d ƒ rª t | d | j j d d ƒ ƒ n x' | j d g ƒ D] } t | | ƒ q½ Wt | d | j d i ƒ ƒ x* | j d
g ƒ D] } t | d | ƒ qW| j d ƒ r‰| j d ƒ r‰i | j d ƒ d 6g | d <| j d
ƒ r‰| j d
ƒ | d d d
s xmlns:planetR\ s xml:langR* R0 R, R3 R- i R R€ Rƒ t publishedt published_parseds
dc_date.takens %s:%st dct
date_Takens %ss xmlns:%ss http://purl.org/dc/elements/1.1/R{ t
feedburnert origLinks %s_%st whereR; t coordinatest Pointi t Boxt
LineStringt Polygont geo_latt geo_longt georss_points [,\s]t georss_linet
georss_circlet
georss_boxi i t georss_polygonR| R R‰ RU R} R~ R‘ RŠ R' t planet_idN( s dcs
date_Takens xmlns:dc( s
feedburnerR™ (% R R_ R` R? t planett xmlnsR. R\ t feedR' R7 Rn R* R R R- R@ RA RJ t
entry_updatedRC R R RK R# t
namespacesRw t floatt ret splitRU RŽ R t __class__t copyR‘ t version( Rª R5 R R4 Rn R— RO t nsR t xoriglinkRš R; R› R| R~ t srcR t
src_authorR ( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt reconstitute s¨
& % & " $&7'''K' "
c C s] | d f | d f | d f f } x5 | D]- \ } } | j | ƒ r( | | r( | | Sq( W| S( NR„ R• ( R. ( Rª R5 t defaultt chkst nodet field( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyR« k s (% t __doc__R® RC t sgmllibt xml.sax.saxutilsR t xml.domR R t html5libR t html5lib.treebuildersR R¨ RE t hashlibR R t compilet UNICODER
R R R R" R& R' R7 RJ RK RU R- Rw R‘ R· RA R« ( ( ( s@ /home/sa3ruby/intertwingly.net/code/venus/planet/reconstitute.pyt s4 $
: ( j