Building bridges - Plone Conference 2015 Bucharest

. . Building Bridges Integrative publishing solutions with Plone. From storages to converters. Andreas Jung @MacYET
 info@zopyx.com
 Plone Conference 2015 Bucharest

/about 20 years in publishing business since 1995 Integrator Building generic and unified solutions Always interested in alternative high-quality components besides the mainstram

Agenda • Storages and services • Integration and federation of external storages and web services into Plone • Documents and formats • converters (A➝B)

Plone as 
 Publishing Platform • Pros • Secure • Workflows • Extensible • Cons • self-contained universe (ZODB) • lack of decent integration with external data sources, cloud storages and cloud services besides relational databases • focused on HTML as content format (in addition to binary data and assets)

A typical publishing workflow

Our Publishing Universe

Our Publishing Universe

(Cloud) storages 
 and web services

External/cloud storages 
 in Plone • Current state: • Reflecto • RDBMS (SQLAlchemy) • Dexterity content only stored in ZODB (no dedicated storage layer) • Archetypes external storages • poor integration story • different integration approaches, different APIs • most add-ons unmaintained

• Plone 4.3/(5.0 compatible) add-on for the integration of other storage systems 
 (other than ZODB) into Plone • Part of our XML publishing toolbox • Can be used without the "XML" stuff • Unified access and API to external storages and services • Available modes • Mounting • Dexterity support • 100 tests, Plone 4.3/5.0 against 6 different storage backends

XML Director - Mounting • Plone "Connector" content-type • parameters: connection URL, username, password • acts as a mountpoint • URL traversal support • ZIP import/export, multi-file upload • basic UI for creating/renaming/deleting collections/ folders and resources • simple view registry • ACEditor integration for common formats • minimal, small and extensible • no indexing support • no proxy object magic as in Reflecto • intended for applications that need to access external data sources and storages

root de en my-onkopedia onkopedia-p knowledge-database mammakarzinom-des-mannes mammakarzinom-der-frau … … onkopedia current archive draft Version 01.04.2013 Version 07.08.2014 Version 25.03.2012 pdf xml html media source 1.jpg 2.jpg … incoming.docx index.html index.xml index.pdf my-onkopedia source incoming.docx xml index.xml html index.html media 1.jpg 2.jpg … pdf index.pdf source incoming.docx xml index.xml html index.html media 1.jpg 2.jpg … pdf index.pdf Connector http://host/de/my-onkopedia/mammakarzinom-der-frau/archive/version-25.03.2014/@@view/xml/index.xml Connector Connector

XML Director - Dexterity • three new Dexterity fields • XMLText (stores, validates XML) 
 with ACEditor widget • XMLImage, XMLFile • XPath • content stored on the configured storage • flat storage hierarchy based on UID • dedicated set() and get() methods (due to lack of a DX storage API) as data managers • DX behaviors not applicable here • (we need a Dexterity storage API or some wrapper in plone.api) xml_text = XMLText() xml_image = XMLImage() obj.set_xml('xml_text', xml) obj.set_xml('xml_img', img_bin) xml = obj.get_xml('xml_text') img_bin = obj.get_xml('xml_img')

pyfilesystem • abstraction layer on top of storages, access through a uniform API • Python 2/3 compatible • various filesystem/webservices drivers • Goal: your code must not know about the underlaying storage system. The backend is just aconfiguration option. • extensible (writing a new driver is straight forward • sandboxed filesystem operations • OOTB support for: WebDAV, S(FTP), RPCFS, OSFS, S3, ZIP, Memory, MultiFS, WrapFS handle = fs.opener(some_url) with handle.open('foo', 'w') as fp: fp.write(data) handle.listdir(dirname) handle.makedir('foo/bar/test') handle.removedir('foo/bar/test) handle.exists(some_filename) handle.isfile(some_name) handle.move(src, dst) handle.copy(src, dst) ….

WebDAV (S)FTP pyfilesystem Plone xmldirector.plonecore Dropbox GDriveAWS S3 Local FS Architecture OwnCloud
 Alfresco
 eXistDB
 BaseX Dropbox Sharepoint Evernote Facebook Flickr Yandex OneDrive many others Driver Driver Driver SMEOtixo DropDav WebDAV native
 protocols native
 protocols Your setup SaaS setup

Storage/
 Web Service self-hosted (Privacy) via external SaaS Bridge (limited privacy?) WebDAV 
 (Owncloud, BaseX,
 eXist-DB, Alfresco, etc.) YES YES Amazon S3 YES YES Local filesystem YES NO Dropbox (YES, auth token issues) YES FTP/SFTP (YES, V1.4) YES 4Shared ADrive Alfresco Amazon Cloud Amazon S3 Box CloudMe Copy Cubby Digital Bucket DriveOnWeb Dropbox Dump Truck Evernote FTP Fabasoft Facebook FilesAnywhere Flickr GMX.DE Google Drive HiDrive Huddle LiveDrive Mediencenter MyDrive OneDrive Online FileFolder OwnCloud Picasa SugarSync TrendMicro SafeSync Web.de WebDAV Yandex NO YES pyfilesystem driver options

Supported services through 
 3rd party services (example)

https://pypi.python.org/pypi/xmldirector.plonecore/1.3.0b1

Document formats 
 and conversion options

Professional
 Publishing Structured data Metadata Structured content Document relations

• Industry standard in publishing • structured data • structured content • not many alternatives besides Indesign stuff…

Advantages of XML • XML structure definition • Document Type Definition 
 (DTD) • XML Schema (XSD) • RelaxNG 
 • XML business rules • Schematron

XML transformations • Transformations • XSLT (version 1-3) • rule based language • Transformation between 
 XML dialects • or Python • or ……

XML 
 transformation pipelines XML 1 XML 2XSLT XSLT XSLT XML 1 XML 2Python XSLT Python

Format: DOCX • DOCX is XML but the same crap as .DOC on a different level • all DOCX converters suck in their own special ways • dedicated Word templates require dedicated converters and special treatment • usually converted to some XML dialect 
 (e.g. Docbook 4/5) • Tools • past: LibreOffice/OpenOffice (HTML) • currently: c-rex.net (dedicated XML schema) • others: Transpect (Le-TeX)

Format: DITA • DITA = Darwin Information Typing Architecture • XML model for authoring • defacto industry standard for technical documentation • focus on content reuse • Information typing: Task, Concept, and Reference • key concepts: topics and maps • extensive metadata and specialization • Tools • DITA toolkit for publishing (HTML, PDF, ODT, Docbook) • XMLMind Ditac

Format: HTML5 • HTML5 as primary source for quality publishing 
 (vs. XML)? ……questionable • semantic elements <article>, <section>, <header>, <figure>, <nav>… • freedom of structure (HTML5) vs. 
 enforced structure and semantics (XML) • not really suitable for professional high-quality publishing
 (seen differently by others) • often used as intermediate format for CSS Paged Media with XML as primary format

Format: PDF (1/2) • traditional: XML ➝ XSL-FO ➝ PDF • CSS Paged Media: HTML + CSS ➝ PDF • Tools (you get what you pay for, better quality=higher price) • WKHMLTOPDF (free), Weasyprint • PDFReactor (RealObjects) • PrinceXML (Prince) • PDFChip (Callas Software) • Antennahouse V6.2 CSS Formatter • Plone integration via Produce & Publish Plone Client Connector, collective.sendaspdf, abstract.wkhtmltopdf, eea.pdf free $$$$

• new project: Vivliostyle (open-source + commercial) • "One Source Multi-Use for making eBooks, Web, and Print books" • based on EPUB Adaptive Layout implementation
 http://www.idpf.org/epub/pgt/ • fixes many limitations of the CSS Paged Media approach and EPUB limitations Format: PDF (2/2)

Format: ODF • ODF is completely irrelevant in the publishing world (DOCX is (still) king) • Tools: • Pandoc • OpenOffice • LibreOffice

Format: TeX/LaTeX • Perfect for text-oriented layouts • unusable for complex layouts • Tools: • ftw.book • Pandoc • Transpect

Format: E-Books (1/2) • different ebook formats: 
 EPUB, EPUB3, Mobi, KF8, Apple's EPUB3 • different hardware and software readers:
 Kindle, iOS & Android devices, Kobo, Toliono, Sony, Nook • fixed format ebooks vs. reflowable ebooks vs. adaptive layouts • many limitations regarding typography, handling of images and tables HUGE MESS

• Tools: • Calibre (Python) • eea.epub • Produce & Publish server (via Calibre) • web services like "Bookalope" Format: E-Books (2/2)

Plone as platform for 
 publishing solutions

www.xml-director.info demo.xml-director.info
 xmldirector.plonecore Questions?