LDP
/
python-tldp
mirror of https://github.com/tLDP/python-tldp
0
1
Fork 0
Code Releases Activity
706 Commits 1 Branch 20 Tags 669 KiB
Commit Graph

407 Commits

Author SHA1 Message Date
Martin A. Brown bd7b08ce23 OK, so a can be anything, not just a directory 2016-03-07 18:54:08 -08:00
Martin A. Brown 87e1161212 add support to driver for --builddir logic 
create the --builddir before building
if --publish, swap the built directory with the output directory
then --remove the old content
 2016-03-07 18:52:13 -08:00
Martin A. Brown b2a8ac28d2 adding an empty attribute called "build" 2016-03-07 15:04:54 -08:00
Martin A. Brown 8c6ceba912 adding a function to swap directories 2016-03-07 15:04:16 -08:00
Martin A. Brown 015b3459f4 more refactoring, heading toward --publish 
created function removeOrphans() and removeUnknownDoctypes()
and the function runbuild() ugly name; which is called from all three of the
main work functions, build(), publish() and script()
 2016-03-07 13:50:15 -08:00
Martin A. Brown dab1fc6bbc abstract error-handling; prepare for --publish 
abstract the error-handling away from the one-large run() function into each
of the functions (show_doctypes, show_statustypes, detail, summary, etc.)
add the function publish(), which will call build() and ensure success before
running any of the publishing
 2016-03-07 13:21:38 -08:00
Martin A. Brown 30d44a75f8 better refactoring of the large "run" method 2016-03-07 12:39:22 -08:00
Martin A. Brown 814dfec181 pep8/pyflakes fixes 2016-03-07 12:12:34 -08:00
Martin A. Brown 76fd27d1fa pep8/pyflakes fixes 2016-03-07 12:10:49 -08:00
Martin A. Brown 517a29b4a8 switch to using os.EX_OK for sys.exit() 
also more preparation for switching to use --builddir
 2016-03-07 12:02:25 -08:00
Martin A. Brown dfc20c5617 move directory-handling logic to the processor 
In preparation for supporting a separate --builddir (allowing minimal
disruption of real output directory during rebuild) factor all output
directory handling logic into the main processor object (BaseDoctype).
Simplify the generate() method.
Centralize all pre-build logic in hook_build_prepare().
Remove all hook logic from the OutputDirectory.
 2016-03-07 11:34:09 -08:00
Martin A. Brown c390e71b4a removing all chdir() and resource copying logic 
the logic for making sure to chdir() into the build directory has been
sequestered into doctypes/common.py (and output.py); additionally, it is
smarter to put the resource copying logic, there, as well
 2016-03-07 10:20:55 -08:00
Martin A. Brown 98b19ac5ce do not mkdir() if we are in --script mode 2016-03-07 10:01:42 -08:00
Martin A. Brown fc4c83307f do not mkdir() if we are in --script mode 2016-03-07 10:01:26 -08:00
Martin A. Brown 99d6232259 simplify text in commented lines 2016-03-07 10:00:51 -08:00
Martin A. Brown f87458c461 rearranging and renaming build setup methods 
in preparation for supporting a build-directory, moving the os.chdir() and the
copying of image files into tldp/doctypes/common.py and adding a few hooks in
the main logic for building
 2016-03-07 09:50:22 -08:00
Martin A. Brown 7308f331ff parameterize the --resources to copy at build time 2016-03-07 09:39:34 -08:00
Martin A. Brown 9c8746e486 and report on the output directory, if present 2016-03-07 09:15:17 -08:00
Martin A. Brown b30e2af282 require exact signature match; stop with .lower() 
was comparing for case insensitive matches when locating signatures; probably
a bad idea; better to simply require an exact match
 2016-03-07 09:11:35 -08:00
Martin A. Brown 853aec028b a bit more info, when --verbose 2016-03-07 09:04:25 -08:00
Martin A. Brown 59bcafb874 add --publish option 
in preparation for separating --publish and --build, add the option
 2016-03-07 09:03:57 -08:00
Martin A. Brown a002fd926d add --publish option 
in preparation for separating --publish and --build, add the option
 2016-03-07 09:00:15 -08:00
Martin A. Brown 0e955fce06 check --builddir/--pubdir on same filesystem 
ensure that both the --builddir and the --pubdir are on the same filesystem
so that we can (reasonably) safely os.rename() after the build is done
 2016-03-07 08:39:34 -08:00
Martin A. Brown bfc8328181 be just a touch more informative about what went wrong 2016-03-07 08:13:50 -08:00
Martin A. Brown 88ee1bf5fa support new option --builddir 2016-03-07 08:06:17 -08:00
Martin A. Brown e6b6ea7b40 add support for --doctypes and --statustypes 
provide CLI-discoverable listing of supported source document types and status
types
 2016-03-06 11:29:13 -08:00
Martin A. Brown bfd6c1a0a1 tldp/doctypes/docbook4xml.py 
correct dependency listings for validated source removal
 2016-03-05 19:24:05 -08:00
Martin A. Brown 760cd392f4 use newer, simpler topo-sort for dependency tracking 2016-03-05 17:08:33 -08:00
Martin A. Brown 4c01ae4af7 simplify topological dependency solution 
This patch prepares the way for simplifying the topological sort solution for
the classes which implement the document building logic.  Formerly, each
doctype class had to import networx itself and the @depends decorator stuffed
the dependencies into a graph in the class variable.

Now, each method tracks its dependencies (same decorator trick), but the
topological sort is not computed until just before running the job.  This is
more flexible, more obvious, simpler and features less code replication.

The next commit or two will convert the remaining doctype classes to use this
techinque.
 2016-03-05 17:04:45 -08:00
Martin A. Brown 7d287b44e5 the required items should say asciidoc_ 2016-03-05 15:58:26 -08:00
Martin A. Brown 428e577c0d try not to go over 80 chars (attempt #3) 2016-03-04 21:44:50 -08:00
Martin A. Brown 93fb5b3356 quote that argument 2016-03-04 21:34:44 -08:00
Martin A. Brown d8f14c9e55 initial support for asciidoc format 2016-03-04 17:10:30 -08:00
Martin A. Brown f617cca3d3 add support for Asciidoc detection 2016-03-04 17:10:21 -08:00
Martin A. Brown 50af88ebde switch to os.path.exists(), prep for chunked is own subdir 2016-03-04 17:07:36 -08:00
Martin A. Brown 1dbc0e5f8b always log the contents of the tldp-build-* files in --debug mode 2016-03-04 17:06:55 -08:00
Martin A. Brown 5f9533045c shrink the total window width (want to fit in 80 cols) 2016-03-04 12:43:28 -08:00
Martin A. Brown 2a0e383c26 run the xinclude first, then validate document 2016-03-04 11:46:17 -08:00
Martin A. Brown 3e1b732b32 correct the help string for docbook.rng 2016-03-04 08:53:53 -08:00
Martin A. Brown 92e7c8bc89 lowering verbosity on reporting tool/data detection 2016-03-04 07:58:03 -08:00
Martin A. Brown bba998b7b9 adding NS XSL fs locations for Debian 2016-03-04 07:53:29 -08:00
Martin A. Brown 0ae4fd4806 flesh out support for DocBook 5.0 
this is the main commit which provides support for generating documents from
sources written in DocBook 5.0.
 2016-03-03 23:45:20 -08:00
Martin A. Brown 9a5bd5fd05 scripts look stupid without a final newline 2016-03-03 23:45:05 -08:00
Martin A. Brown c8f489ef5d reset logging level after parsing all configs 2016-03-03 14:03:38 -08:00
Martin A. Brown 7359af070d adjust and reorganize imports 2016-03-03 11:27:19 -08:00
Martin A. Brown b092e09516 add support for --script; shorten names 
adding support for the --script option
shortening the names in the script since it was getting fairly redundant to
keep typing tldp.outputs.Output*, tldp.inventory.Inventory and
tldp.sources.Source*
 2016-03-03 11:23:54 -08:00
Martin A. Brown 0ee742b64b add internal support for dumping shellscript 
added support to the BaseDoctype class so that it can dump a script instead of
executing the script
 2016-03-03 11:22:09 -08:00
Martin A. Brown e2aff4f388 add alternate constructor 2016-03-03 11:21:54 -08:00
Martin A. Brown 83e1c7a57b needed output.validsource for DocBook XML xmllint validation 2016-03-02 23:52:50 -08:00
Martin A. Brown 6ac2699fbf removing reference to unused variable buildorder 2016-03-02 23:52:29 -08:00
Martin A. Brown 369d53eb35 be louder about the signature finding (or not) 2016-03-02 23:51:27 -08:00
Martin A. Brown 17772e948c call xmllint before doing any other work 
important to validate the document before processing it;
Note!  Adding this feature flushed out a number of sketchy corners of a few of
our documents, but almost everything validated already.
 2016-03-02 23:50:19 -08:00
Martin A. Brown 2094199462 pyflakes/pep8 fixes 2016-03-01 22:58:37 -08:00
Martin A. Brown ee88d451a8 pyflakes/pep8 fixes 2016-03-01 22:53:07 -08:00
Martin A. Brown 6e11db3957 add a test case for when user specifies only stem as arg 2016-03-01 22:46:57 -08:00
Martin A. Brown 7f26728ec1 say "source" documents instead of "total" (less confusing) 2016-03-01 22:41:16 -08:00
Martin A. Brown 9cb92dce7c build() only returns 0 (success) for all builds 
before, the build() function lacked the logic to retain, and return that there
was a failure during the building of one of the documents; it now returns
a 1, which is passed directly to sys.exit(), if any single build fails
 2016-03-01 22:21:29 -08:00
Martin A. Brown 97181b167a temporarily forgetting about other doctypes 2016-03-01 21:50:58 -08:00
Martin A. Brown d28460aaff bump up verbosity of failed excute() to logger.info 2016-03-01 21:38:16 -08:00
Martin A. Brown efa2c9432a bump up verbosity of failed excute() to logger.info 2016-03-01 21:38:04 -08:00
Martin A. Brown bf99f97eba print out detail files in sorted order 2016-03-01 21:37:42 -08:00
Martin A. Brown 51250d4a47 add a tiny bit more verbosity to output detail 2016-03-01 20:17:23 -08:00
Martin A. Brown 4939347e85 adjust verbose output a tad 2016-03-01 20:12:01 -08:00
Martin A. Brown 17ffc0434f make the sought text more obvious in logging output 2016-03-01 19:51:19 -08:00
Martin A. Brown 76d5886e54 fix skipping SCSI-2.4-HOWTO (bad stem/ext handling) 
source document identification was skipping any documents which lived in their
own directory AND whose names had a dot; fixed and test support added
 2016-03-01 19:49:28 -08:00
Martin A. Brown 08ef4c35bf only say we are removing, if removing 2016-03-01 18:38:58 -08:00
Martin A. Brown dd6cda7726 improve diagnostic error messages a little bit 2016-03-01 14:32:14 -08:00
Martin A. Brown dfa729b9e5 restore driver.py to functionality 
after refactoring into a bunch of separate functions, the tests finally pass
again
 2016-03-01 14:10:51 -08:00
Martin A. Brown ae89d6814a inverted logic; caught in testing code 2016-03-01 13:36:09 -08:00
Martin A. Brown 2feb454c18 create more functions for the driver.py 
Rearrange the logic for handling the various arguments so that it is
less obnoxious; breaking it into functions makes it easier to read (and test).
 2016-03-01 13:21:26 -08:00
Martin A. Brown 4ec06998b4 correct the labeling in the --help output 2016-03-01 13:21:11 -08:00
Martin A. Brown 7664d35090 use abspath before looking for documents 2016-03-01 11:35:06 -08:00
Martin A. Brown b54962900d allow case-insensitive matching for document types 
The Python names for the document types are upper-cased and probably not
predictably named for the end user; allow a case-insensitive match when trying
to turn off building or listing for a particular document type.
 2016-03-01 11:01:32 -08:00
Martin A. Brown aff926bb58 improve the handling of --skip 2016-03-01 10:30:18 -08:00
Martin A. Brown fe54af73a5 add default (system) configuration file 2016-03-01 09:04:45 -08:00
Martin A. Brown 8792b4c83a make individual assertions 2016-03-01 08:20:57 -08:00
Martin A. Brown 6e2041692e skip orphaned documents passed into build() 2016-03-01 07:31:52 -08:00
Martin A. Brown c4ee8366f9 reporting when skipping a dir/file in sourcedir 2016-03-01 07:31:19 -08:00
Martin A. Brown 388215186e adding main() as entry point 2016-03-01 07:15:35 -08:00
Martin A. Brown bb14a8cb51 allow file to be specified for summary 
also allow the creation of an in-memory OutputDirectory object before build
 2016-02-29 21:32:20 -08:00
Martin A. Brown dfc9855991 logic overhaul; abstract inventory creation 
each function was creating its own inventory; this is not necessary
also, if a user wants only to build a single document, it is not necessary to
scan the entire source and output directory (small, but perceptible speedup to
the end user)
better checking of the various different possible argument types, since an
argument can be a filename, a status_class or a stem
 2016-02-29 20:33:33 -08:00
Martin A. Brown 16885508da pep8/pyflakes 2016-02-29 20:33:14 -08:00
Martin A. Brown 363bba8a37 pep8/pyflakes 2016-02-29 20:32:36 -08:00
Martin A. Brown 56b1a027ca access inventory by status class name 
create method getByStatusClass() and add testing
 2016-02-29 18:03:16 -08:00
Martin A. Brown d807c02173 create and use general function arg_issourcedoc 
a source document is defined as either a file with a known ending (e.g. .xml,
.sgml) or a directory name containing a file with the directory name plus a
known extension

create a small function (arg_issourcedoc), which takes a filename and returns
that filename if it is (possibly) a source document or, if it is a directory,
finds the main file in the directory which represents the main document
 2016-02-29 17:59:49 -08:00
Martin A. Brown 17a60e367a make the action --options a mutually exclusive group 2016-02-29 17:56:41 -08:00
Martin A. Brown fa73f93080 messed up indentation 2016-02-29 11:37:13 -08:00
Martin A. Brown ddb5424941 get 100% coverage on the utils.py module 2016-02-29 11:32:01 -08:00
Martin A. Brown 6cdf0b65c1 explicitly stringify values at "input" 
user can pass in a dict(); os.environ always contains string values, though,
so make sure that the dict() looks like what would be returned from os.environ
 2016-02-29 10:48:03 -08:00
Martin A. Brown 98dd5ecdf8 no config validation yet 
(and, I should not have used such terms inline in code)
 2016-02-27 11:29:05 -08:00
Martin A. Brown 36e92b83d9 perform the processor prebuild check first 
it is smarter to perform the non-destructive check first, rather than blowing
away the output directory and then reporting that it is impossible to rebuild
the document output tree
 2016-02-27 11:13:58 -08:00
Martin A. Brown fe520cd583 add logging logic to driver.py 2016-02-27 10:52:35 -08:00
Martin A. Brown 5f11427f38 remove pipe from script 2016-02-27 10:46:18 -08:00
Martin A. Brown 55cc7d5775 improve visual grouping of --help output 
use argparse's ArgumentGroups to separate the argparse options for each of the
processors
 2016-02-27 09:45:32 -08:00
Martin A. Brown eac92b5354 add the document stem to the __repr__ 2016-02-26 23:18:50 -08:00
Martin A. Brown 066f319156 change wrapped function name to "timing" for log entry 2016-02-26 23:18:24 -08:00
Martin A. Brown ee68a453d2 exit 1 and report extra args on CLI 2016-02-26 23:18:02 -08:00
Martin A. Brown 9137946a05 adding timing to a few methods 2016-02-26 23:02:00 -08:00
Martin A. Brown e6833f3989 and comma-separated in --verbose mode, too 2016-02-26 22:22:00 -08:00
Martin A. Brown 57055e582d better production of status list 2016-02-26 22:19:30 -08:00
Martin A. Brown 1c4f283482 handle the images and resources during publication 2016-02-26 22:18:02 -08:00
Martin A. Brown de514fd41b user can now specify file, stem or nothing for build 2016-02-26 19:07:05 -08:00
Martin A. Brown 39c4a13d88 repair logic for when there is no index.sgml 2016-02-26 19:04:51 -08:00
Martin A. Brown f739e365c0 do not bother generating index.sgml unless desirable/necessary 2016-02-26 18:15:27 -08:00
Martin A. Brown 9ffdd1df5d support SourceDocument as directory nam 
allow the driver to use the new feature of sources.py and SourceDocument, so
user can refer to simply a directory as the document
 2016-02-26 18:04:39 -08:00
Martin A. Brown e7c85fb2ec allow construction of SourceDocument with file or directory 2016-02-26 13:22:20 -08:00
Martin A. Brown f40ff2f915 unused function 2016-02-26 12:21:04 -08:00
Martin A. Brown f1bc2121a6 moving hook_build_{success,failure} to BaseDoctype 2016-02-26 12:08:05 -08:00
Martin A. Brown 80d692efba massive simplification of generate() method 
move hook_build_success() and hook_build_failure() into BaseDoctype;
subclasses can always override, if need be
 2016-02-26 12:06:34 -08:00
Martin A. Brown 89d8f14231 pyflakes fix 2016-02-26 12:06:27 -08:00
Martin A. Brown 3a0bb65961 switch to the DAG model 2016-02-26 12:06:17 -08:00
Martin A. Brown 84477c0f45 move cleanup() method to BaseDoctype 2016-02-26 11:58:56 -08:00
Martin A. Brown 66f11e5f3b move cleanup() method to BaseDoctype 2016-02-26 11:58:43 -08:00
Martin A. Brown 5602ab68dd pep8 fixes 2016-02-26 11:56:28 -08:00
Martin A. Brown ca43130a6d spit out the contents of the STDERR, STDOUT and executed script if failure 2016-02-26 11:24:42 -08:00
Martin A. Brown c5e14d6ccf "all" is a good default for --list 2016-02-26 11:24:19 -08:00
Martin A. Brown 9d1f066085 better PDF generation logic; add cleanup() 
try to make the PDF first with jw, but don't quit if that fails
fall back to PDF generation with dblatex and tell the user
cleanup any generated index.sgml (if we made it)
 2016-02-26 11:22:00 -08:00
Martin A. Brown 06194feba8 build sequence resilience/terseness) improvements 
it would be a problem if there's no self.required; don't protect against it
be clearer about which methods in which class are being called (debug)
do not fail with AssertionError if missing a tool, just skip to next doc
always call the processor AND output hook_build_{failure,success} methods
 2016-02-26 11:19:30 -08:00
Martin A. Brown 62f96f3bcb add a postamble, too (extra newlines, only) 2016-02-26 10:10:45 -08:00
Martin A. Brown f112fc98d1 switch to the DAG model 2016-02-26 10:10:29 -08:00
Martin A. Brown 2b65730f25 adjusting logging verbosity 2016-02-26 01:07:09 -08:00
Martin A. Brown 4e76494cce rework the build sequence 
adapt all of the methods to the new topological-sort driven technique
 2016-02-26 01:02:16 -08:00
Martin A. Brown 74bdfcd44f improve logging a little bit 2016-02-26 01:00:43 -08:00
Martin A. Brown 85b2b15c3b build a graph of method dependencies 
to allow individual subclasses of BaseDoctype to be able to implement as many
methods as they want to create document outputs, the BaseDoctype now
implements a graph of method dependencies.  The sequence of tasks is
determined by applying the @depends decorator to the methods in the
heavy-lifting classes which actually implement the document procsesing.

For example, the Linuxdoc class will probably have about 6 or 7 methods to
generate the outputs, whereas, the more complex DocbookSGML class will have at
least two or three more to handle the build-time index-generation (if
necessary for the document).
 2016-02-26 00:30:37 -08:00
Martin A. Brown 2d052e443e --skip option needs to have an empty list default 2016-02-26 00:30:05 -08:00
Martin A. Brown 65a80f1339 add __name__ to getLogger() calls 2016-02-25 12:29:55 -08:00
Martin A. Brown bb6fef9635 implement skipping logic in build (driver.py) 2016-02-25 12:29:38 -08:00
Martin A. Brown 44fc44d15a remove logging logic from utils 2016-02-25 12:29:12 -08:00
Martin A. Brown 595e610213 adding --skip feature 2016-02-25 12:28:41 -08:00
Martin A. Brown 859b7f893f pull logger straight from logging; use abs imports 2016-02-25 11:39:18 -08:00
Martin A. Brown e74197eb01 pull logger straight from logging; add BaseDoctype 
rely directly on the logging module directly instead of through tldp.utils
add the BaseDoctype to all of the modules (even though not all are
implemented)
 2016-02-25 11:37:14 -08:00
Martin A. Brown b86f92f742 inspect/call the argparse method for each doctype 2016-02-25 10:38:38 -08:00
Martin A. Brown 81375b84c5 revert to name knowndoctypes (classes only) 
no need to itemize knowndoctypemodules after moving the config_fragment
functions to the staticmethod argparse on each class
 2016-02-25 10:37:45 -08:00
Martin A. Brown 7903d5f497 config_fragment becomes staticmethod in class 
the config_fragment function becomes a part of each doctype class, allowing
better encapsulation of most/all document type handling logic
the config_fragment function becomes a staticmethod called argparse
 2016-02-25 10:35:46 -08:00
Martin A. Brown 92a9042a74 add automatic detection of config_fragments 2016-02-25 09:58:11 -08:00
Martin A. Brown eb4e3d8c38 inspect for classes and modules 2016-02-25 09:51:17 -08:00
Martin A. Brown 03881d20de use case-insensitive sorting 2016-02-25 09:31:49 -08:00
Martin A. Brown db28d83480 decrease logging yak; remove unused functions 2016-02-24 09:26:23 -08:00
Martin A. Brown 89cb766ad7 clean up the stray index.sgml left hanging around 2016-02-23 23:30:07 -08:00
Martin A. Brown 4c7e92ca17 a wee bit more logging and better emphasis on the word SUCCESS 2016-02-23 22:54:57 -08:00
Martin A. Brown 7db6bb842e add an asterisk for log readers 2016-02-23 22:44:39 -08:00
Martin A. Brown fc2cdf975d switch all processors to generated shell scripts 2016-02-23 22:44:17 -08:00
Martin A. Brown ce359dfa32 generic method shellscript() cannot test if job is done 2016-02-23 22:37:59 -08:00
Martin A. Brown 8630157046 adapt to generated shellscript model 2016-02-23 21:59:47 -08:00
Martin A. Brown c6ebbf8a64 make a bit less noise in routine operation 2016-02-23 21:59:20 -08:00
Martin A. Brown 120d44ffa5 make a bit more noise when building and skipping 2016-02-23 21:58:56 -08:00
Martin A. Brown ad2a1763ec move method shellscript into BaseDoctype 
and fix a few errors in the generated DocBook XML 4.x script
 2016-02-23 21:58:03 -08:00
Martin A. Brown f1343deb00 try out a shellscript writing technique instead 2016-02-23 21:17:30 -08:00
Martin A. Brown 7da0670d23 add just a bit more warning bells 2016-02-23 21:17:01 -08:00
Martin A. Brown 83b998cb51 trying to write and execute a shell script 2016-02-23 21:16:44 -08:00