Martin A. Brown
03881d20de
use case-insensitive sorting
2016-02-25 09:31:49 -08:00
Martin A. Brown
db28d83480
decrease logging yak; remove unused functions
2016-02-24 09:26:23 -08:00
Martin A. Brown
89cb766ad7
clean up the stray index.sgml left hanging around
2016-02-23 23:30:07 -08:00
Martin A. Brown
4c7e92ca17
a wee bit more logging and better emphasis on the word SUCCESS
2016-02-23 22:54:57 -08:00
Martin A. Brown
7db6bb842e
add an asterisk for log readers
2016-02-23 22:44:39 -08:00
Martin A. Brown
fc2cdf975d
switch all processors to generated shell scripts
2016-02-23 22:44:17 -08:00
Martin A. Brown
ce359dfa32
generic method shellscript() cannot test if job is done
2016-02-23 22:37:59 -08:00
Martin A. Brown
8630157046
adapt to generated shellscript model
2016-02-23 21:59:47 -08:00
Martin A. Brown
c6ebbf8a64
make a bit less noise in routine operation
2016-02-23 21:59:20 -08:00
Martin A. Brown
120d44ffa5
make a bit more noise when building and skipping
2016-02-23 21:58:56 -08:00
Martin A. Brown
ad2a1763ec
move method shellscript into BaseDoctype
...
and fix a few errors in the generated DocBook XML 4.x script
2016-02-23 21:58:03 -08:00
Martin A. Brown
f1343deb00
try out a shellscript writing technique instead
2016-02-23 21:17:30 -08:00
Martin A. Brown
7da0670d23
add just a bit more warning bells
2016-02-23 21:17:01 -08:00
Martin A. Brown
83b998cb51
trying to write and execute a shell script
2016-02-23 21:16:44 -08:00
Martin A. Brown
cf8c314981
--pubdir type checker should be an arg_isdirectory
2016-02-23 13:13:52 -08:00
Martin A. Brown
8c69e05f94
beefing up some testing for new functions
2016-02-23 13:10:10 -08:00
Martin A. Brown
2fef28a299
adjust the logging information
2016-02-23 12:25:54 -08:00
Martin A. Brown
6ad64ea0e0
add an arg_isfile
2016-02-23 12:18:46 -08:00
Martin A. Brown
d5524f09ee
move generic function into BaseDoctype
2016-02-23 12:18:33 -08:00
Martin A. Brown
80011bcf9d
move generic function into BaseDoctype
2016-02-23 12:18:17 -08:00
Martin A. Brown
3788c4d265
get rid of the paths in mabrown home dir
2016-02-23 12:16:17 -08:00
Martin A. Brown
53c91eebe6
rename the hooks so they start with hook_
2016-02-23 11:58:29 -08:00
Martin A. Brown
a415484b5a
separate operations by method and dependencies
...
use the generalized build sequence running supported by doctypes/common.py
2016-02-23 11:57:37 -08:00
Martin A. Brown
df75f051db
generalizing build sequence logic
...
ready for use in first doctype (linuxdoc)
2016-02-23 11:56:58 -08:00
Martin A. Brown
08df7acbe2
rework tracking of success vector
2016-02-23 11:08:04 -08:00
Martin A. Brown
1475cf7890
use arg_ checking functions; add build_precheck
...
rename platform_check to build_precheck
refer to arg_ checking functions from utils
ensure that just before each build we validate executables
avoid FD leakage by using with ... context for outf (stdout)
2016-02-23 11:05:28 -08:00
Martin A. Brown
11834a17cc
initial work on supporting build verb
2016-02-23 11:05:04 -08:00
Martin A. Brown
b4895a2e0b
use arg_ checking functions from utils.py
2016-02-23 11:04:43 -08:00
Martin A. Brown
58246eb6e7
add config.py to the tldp package
2016-02-23 11:04:20 -08:00
Martin A. Brown
6c4fd88944
adding arg_* handlers; fix FD leakage in execute
...
fix FD leakage in function execute: must close stdout and stderr if
execute opens them (process logging in per-document logdir)
add functions arg_isloglevel, arg_isdirectory and arg_isexecutable
2016-02-23 11:02:36 -08:00
Martin A. Brown
f9453185df
deleting unnecessary platform.py file
2016-02-23 09:44:12 -08:00
Martin A. Brown
51fabb65d7
change platform to config
2016-02-23 09:43:27 -08:00
Martin A. Brown
13bb8d3129
remove reference to platform
2016-02-23 09:43:06 -08:00
Martin A. Brown
3a0ee539fa
adding [%(default)s] to each argparse entry
2016-02-23 09:08:08 -08:00
Martin A. Brown
5508d1113b
repair detail() and status() generation functions
...
and pass args into all driver functions
2016-02-23 09:00:27 -08:00
Martin A. Brown
f567a0be13
add status_classes for user interaction
2016-02-23 08:59:56 -08:00
Martin A. Brown
04c10a6af9
refer to %(default)s; shorter lines
2016-02-23 08:59:37 -08:00
Martin A. Brown
7bb5e35aeb
add detail method to source and output documents
...
the detail method produces some output listing about the
document and will soon include more verbosity
2016-02-23 08:44:16 -08:00
Martin A. Brown
abb471c912
refer to status_classes and pep8 fix
2016-02-23 08:44:05 -08:00
Martin A. Brown
5dee57763c
pep8 fix
2016-02-23 08:43:44 -08:00
Martin A. Brown
d3be338b75
begin logic for user interaction in driver.py
2016-02-22 21:22:42 -08:00
Martin A. Brown
4fb18c8822
rename sources/outputs to singular; remember newer files
2016-02-22 21:22:17 -08:00
Martin A. Brown
01acf2dfa6
add a property/function for reporting missing files
2016-02-22 21:21:48 -08:00
Martin A. Brown
c821eadac1
improve command-line utility; work out the kinks (before testing)
2016-02-22 21:21:24 -08:00
Martin A. Brown
ce7f703929
itemize valid types for status codes
...
and standardize on name 'orphan' (not 'orphans' or 'orphaned')
assume that SourceCollection and OutputCollection keys are always sorted
2016-02-22 20:24:19 -08:00
Martin A. Brown
b62b732a23
correct location of logging.debug message, suppress None in cfg
2016-02-22 20:23:59 -08:00
Martin A. Brown
25add163cb
rely on LDPDocumentCollection
2016-02-22 20:02:12 -08:00
Martin A. Brown
fb0001a397
adding commonal parent class LDPDocumentCollection
...
factored out the dict-like behaviour (and added sorting behaviour)
from SourceCollection and OutputCollection to LDPDocumentCollection
2016-02-22 20:00:24 -08:00
Martin A. Brown
6fe55dc912
and deal with the problem ofg None
2016-02-22 18:03:44 -08:00
Martin A. Brown
b6edf39aeb
try the loglevel before using it
2016-02-22 17:46:41 -08:00
Martin A. Brown
2087719782
removing extraneous sample invocations
2016-02-22 17:28:15 -08:00
Martin A. Brown
15fe8fc3b6
factor out --dump-cli and --dump-env logic into functions
2016-02-22 17:27:27 -08:00
Martin A. Brown
f6eedfcb7b
flesh out CLI options and add --loglevel option
2016-02-22 17:10:45 -08:00
Martin A. Brown
7486a26cc3
initial commit of main 'ldptool' driver program
2016-02-22 16:20:15 -08:00
Martin A. Brown
82f5b63006
make CascadingConfig behave more like ArgumentParser
2016-02-22 16:19:05 -08:00
Martin A. Brown
cfd54d4524
pep8 fixes
2016-02-22 13:04:26 -08:00
Martin A. Brown
4e07f2a52f
absolute imports (and move isdirectory to utils)
2016-02-22 13:04:16 -08:00
Martin A. Brown
0a32e87dfc
moving isdirectory into utils
2016-02-22 13:04:03 -08:00
Martin A. Brown
bfd07eb4d5
remove superlong opts, make --help output prettier
2016-02-22 12:55:35 -08:00
Martin A. Brown
8b7c4a574c
adding the main config module for the application
2016-02-22 12:34:26 -08:00
Martin A. Brown
3f37d09fe9
add configurables and discovery of files
...
add argparse-style support to each doctype module, for path-configurables
also add the runtime discovery of files needed for document handling (e.g. XSL
and DSSSL files)
2016-02-22 12:32:35 -08:00
Martin A. Brown
468f0f6317
adding firstfoundfile function
2016-02-22 12:32:16 -08:00
Martin A. Brown
c1c4b2b79c
separate init and config resolution; dump configs
...
separate the initialization process from the resolution
of configurations
add logic to dump configuration file formatted outputs, as well
2016-02-22 12:30:34 -08:00
Martin A. Brown
a959db5485
initial configuration snippets
2016-02-21 17:49:14 -08:00
Martin A. Brown
372ae734de
adding some logic for repproducing received configurations
2016-02-21 17:48:46 -08:00
Martin A. Brown
e7af014a73
initial commit of cascading config
...
this utility allows resolution of configuration data from multiple sources;
for example, 'compiled-in' defaults, system configuration file, user
configuration file, process environment and, of course, command-line options
2016-02-21 12:19:51 -08:00
Martin A. Brown
a88451c124
report number of files involved
2016-02-20 20:10:35 -08:00
Martin A. Brown
19f01a0a4c
begin refactoring of directory document handling
2016-02-19 00:55:53 -08:00
Martin A. Brown
b1840b03bf
record the 'broken' field in sdoc/odoc
...
and add function to list_broken (analogically appropriate)
2016-02-19 00:54:39 -08:00
Martin A. Brown
f200c0efc2
add function stem_and_ext to the utils
2016-02-19 00:54:16 -08:00
Martin A. Brown
d7fac8d65f
move logic from SourceCollection to scansourcedirs
...
moving the source dir scanning logic into a function (in preparation for
further refactoring of single-file or entire-directory source document
detection)
adapting tests (by changing the name from SourceCollection to scansourcedirs).
no other tests required
added new test to ensure that an empty SourceCollection() returned as expected
2016-02-18 23:31:18 -08:00
Martin A. Brown
3ddb77b55e
adding some docstrings
2016-02-18 23:07:53 -08:00
Martin A. Brown
6762743f66
docstrings for functions
2016-02-18 23:07:44 -08:00
Martin A. Brown
f04b2dfbd0
pep8 fix
2016-02-18 23:07:14 -08:00
Martin A. Brown
1578e282cc
ensure that statfiles includes directories
...
statfiles was not including directories; it does now
2016-02-18 21:55:56 -08:00
Martin A. Brown
88f518ede7
add docstrings and text to utils functions
2016-02-18 17:23:26 -08:00
Martin A. Brown
7d3843c535
adding a bunch of docstring docs
2016-02-18 13:58:53 -08:00
Martin A. Brown
28f7a8468c
minor documentation fixes
2016-02-18 13:58:30 -08:00
Martin A. Brown
809ddc545b
adding # -*- coding: utf8 -*-
2016-02-18 13:25:02 -08:00
Martin A. Brown
bf768d0277
adding basic docstrings
2016-02-18 13:22:48 -08:00
Martin A. Brown
605b57a1ea
sorted(), so dirs and docs are processed stably
...
use sorted() on the sourcedirs and the contents of each directory so that the
directories are always handled in order and the documents are also handled in
order
adjust logging also to refer to "Source collection dir" rather than just "dir"
2016-02-18 09:17:25 -08:00
Martin A. Brown
20ae6bd25c
add stem to logging entries
...
for consistency, make sure that stem is included on logging lines
2016-02-18 09:16:45 -08:00
Martin A. Brown
a34738301f
removing extraneous logging entry
2016-02-18 09:16:37 -08:00
Martin A. Brown
d643883e50
change name_index to name_indexhtml; adapt tests
...
also minor logging adjustments for clarity and consistency
2016-02-18 09:15:47 -08:00
Martin A. Brown
62198e5973
in execute, before Popen, ensure cmd[0] has +x
...
rename is_executable() to isexecutable()
adjust testing code to match change to execute()
2016-02-18 09:13:46 -08:00
Martin A. Brown
2550047d23
pep8/pyflakes corrections
2016-02-17 19:38:27 -08:00
Martin A. Brown
ef355ad8fb
add inventory category for broken outputs
2016-02-17 19:30:25 -08:00
Martin A. Brown
d3b6ad8e56
add iscomplete property to OutputDirectory
2016-02-17 19:19:48 -08:00
Martin A. Brown
ffc0ad6547
improve readability, parallelism of methods by inverting logic
2016-02-17 19:05:23 -08:00
Martin A. Brown
1039dea0e6
let the prebuild_hook call clean
2016-02-17 19:04:51 -08:00
Martin A. Brown
eed25efbba
simplify cleaning; add (unused) name_epub
2016-02-17 19:04:21 -08:00
Martin A. Brown
f5a65cf843
put stem in logging like many other logging lines
2016-02-17 19:03:37 -08:00
Martin A. Brown
fd3b931423
include hooks for prebuild and postbuild states
2016-02-17 18:31:51 -08:00
Martin A. Brown
bc810dd36c
deal with the ENOENT problem in statfiles()
2016-02-17 18:16:20 -08:00
Martin A. Brown
a0de081099
minor logging tweaking
2016-02-17 17:56:42 -08:00
Martin A. Brown
54a8477a33
renaming to inventory.py (and affected tests)
2016-02-17 16:39:42 -08:00
Martin A. Brown
c425c2a8e8
completed first round of testing of driver code
2016-02-17 13:50:55 -08:00
Martin A. Brown
2bcb55e2a0
flipping the stem/dirname arguments
2016-02-17 13:50:06 -08:00
Martin A. Brown
2b0b241019
adjusting format names
2016-02-17 13:49:33 -08:00
Martin A. Brown
a8284f3afa
repair mtime comparison logic
...
repair mtime comparison logic for source and output directories after
generalizing the statfiles() function in utils.py
add a __repr__ to the Inventory object
2016-02-17 12:04:37 -08:00
Martin A. Brown
46edc205e0
factor out the call to os.stat(); broken links
2016-02-17 11:31:08 -08:00
Martin A. Brown
9301a54ab2
switch to using statfiles
2016-02-17 11:19:48 -08:00
Martin A. Brown
515595cac9
improve statfiles, deprecate ill-conceived fileset
...
statfiles now returns a dict() of statinfo and can handle relative paths
2016-02-17 11:18:47 -08:00
Martin A. Brown
f72583fca6
improve and generalize statfiles and getfileset
...
statfiles is the heavy lifter; call os.stat() while walking the directory
simplify getfileset; use the keys returned in the statinfo dict()
2016-02-17 10:17:19 -08:00
Martin A. Brown
01be89c949
pep8 fixes
2016-02-17 09:12:07 -08:00
Martin A. Brown
c99dbefa92
shorter __repr__ can fit on one line
2016-02-17 08:35:53 -08:00
Martin A. Brown
c1ccc544a3
adjust logic into an inventory object
2016-02-17 08:35:36 -08:00
Martin A. Brown
bc1c154108
inital commit of driver file for handling the porcelain logic
2016-02-17 00:22:31 -08:00
Martin A. Brown
dab2f1f8b1
adding support for documents to know their status
2016-02-17 00:17:49 -08:00
Martin A. Brown
8a1afb6e24
allow creation of empty OutputCollection; fixes
...
Allow creation of an empty OutputCollection so that the object can be handed
around in driver.py for higher-level document wrangling.
Also, repair one inobvious statement, thank you pep8/pyflakes
2016-02-16 23:43:07 -08:00
Martin A. Brown
23a5e40d52
shortening knowndoctypes to kdt
2016-02-16 23:42:57 -08:00
Martin A. Brown
6e3362b9aa
a bunch of filesystem stat()ing functions
2016-02-16 23:42:22 -08:00
Martin A. Brown
f17d164b52
allow creation of empty SourceCollection; fixes
...
Allow creation of an empty SourceCollection, which can be handed around in the
driver to allow for higher-level document wrangling
fix bad, always-failing directory check (thank you, testing)
clarify handling of documents living in a directory and the generation of the
fileset
2016-02-16 23:40:09 -08:00
Martin A. Brown
f39237d307
removing commented block that lives in superclass, BaseDoctype
2016-02-16 23:39:56 -08:00
Martin A. Brown
9845e90a39
adding testing for execute and new function getfileset
2016-02-16 19:52:07 -08:00
Martin A. Brown
fbb6e56929
fixes found during testing
...
should standardize on IOError for errno.ENOENT across the package
adhere to standard choice of logging message
make sure to skip a non-directory in OutputCollection ("continue")
2016-02-16 14:13:17 -08:00
Martin A. Brown
ac44f5d577
refactor detection loop and identify duplicates
...
the nesting was deeper than necessary, so adjusting the detection of files
(and directories) and adding a bit more logging upon duplicate detection
2016-02-16 10:44:24 -08:00
Martin A. Brown
f6f6d4b543
only guess the doctype once
2016-02-16 00:23:58 -08:00
Martin A. Brown
547cb303f6
pep8 and pyflakes fixes (and variable renaming)
2016-02-16 00:03:58 -08:00
Martin A. Brown
5e05b0f30d
starting to wear the belt and suspenders
2016-02-15 23:58:50 -08:00
Martin A. Brown
82a0fce575
initial test script for outputs.py
2016-02-15 23:52:52 -08:00
Martin A. Brown
7b08ececf4
renaming OutputDirs more appropriately to OutputCollection
2016-02-15 23:52:08 -08:00
Martin A. Brown
8163efa0a5
converting OutputTree to behave like a dict
2016-02-15 21:55:58 -08:00
Martin A. Brown
55ef688015
adjust SourceDirs to behave like a dictionary
2016-02-15 21:51:56 -08:00
Martin A. Brown
01bee4a269
adjust error-raising invocations (and tests)
2016-02-15 21:32:35 -08:00
Martin A. Brown
ae189e0d83
adjusting some logging and exceptions for verbosity/clarity
2016-02-15 21:15:29 -08:00
Martin A. Brown
68e3b2f0cb
improving cleaning, esp. index.html
2016-02-15 21:04:41 -08:00
Martin A. Brown
cc6caec3a4
improving readability and simplicity
2016-02-15 20:59:49 -08:00
Martin A. Brown
f9aa61fa32
revise BaseDoctype to add generic generation method
2016-02-15 20:59:25 -08:00
Martin A. Brown
7daefd03bd
renaming Sources to SourceDirs
2016-02-15 20:58:44 -08:00
Martin A. Brown
61f2538927
first working example of Linuxdoc processor
2016-02-15 14:04:09 -08:00
Martin A. Brown
73305ca97f
adding function execute() to shell out
2016-02-15 14:03:56 -08:00
Martin A. Brown
3f08105232
pep8/pyflakes fixes
2016-02-15 14:03:35 -08:00
Martin A. Brown
b6076fec82
create OutputDirectory and OutputTree
...
an OutputTree must exist already, contains all of the OutputDocuments
an OutputDocument determines the stem from the dirname and depends on
the OutputNamingConvention for choosing names for the primary files of each
generated output type
2016-02-15 14:01:55 -08:00
Martin A. Brown
f1c40c1e8c
create BaseDoctype with features needed by all
...
each document processor will need to know about its
sources, outputs and the platform details
the generate() method is a temporary model for operation
2016-02-15 13:58:32 -08:00
Martin A. Brown
169d6872f6
pull some of the key names in by default when "import tldp"
2016-02-15 13:57:48 -08:00
Martin A. Brown
82a8a21d18
check for plain file type, too
2016-02-12 23:59:13 -08:00
Martin A. Brown
fe507461e8
found another typo while testing
2016-02-12 23:49:04 -08:00
Martin A. Brown
236ae3cbd4
skip files with no extension (and complain)
2016-02-12 23:14:51 -08:00
Martin A. Brown
f76aa736e7
when calling type(), the word type is already present
2016-02-12 23:00:54 -08:00
Martin A. Brown
ad73629413
returning the wrong answer... get the FIRST answer (yay for testing)
2016-02-12 23:00:30 -08:00
Martin A. Brown
85e6441fa5
initial impl of class to hold platform-specific details
2016-02-12 13:25:58 -08:00
Martin A. Brown
a7a0fe38b3
adding two unused doctypes
2016-02-12 13:25:19 -08:00
Martin A. Brown
85b8df5e86
method .index removed from Python3 string module; not needed anyway
2016-02-12 13:24:55 -08:00
Martin A. Brown
7affd10e0c
correct the reference to the renamed guess function
2016-02-12 13:24:21 -08:00
Martin A. Brown
9a75657c1e
be more like Python3
2016-02-12 13:24:09 -08:00
Martin A. Brown
7c70a7c4b8
formatname must be a string
2016-02-12 13:00:11 -08:00
Martin A. Brown
92fdf8bec1
case-insensitive sorting is preferred
2016-02-12 12:53:30 -08:00
Martin A. Brown
5263a3a219
fleshing out the linuxdoc processing commands
2016-02-12 12:49:50 -08:00
Martin A. Brown
4652345b7a
adding basic required methods
2016-02-12 12:43:45 -08:00