Download Arelle red hat distribution from http://arelle.org/downloads
http://arelle.org/downloads/16
Install Arelle:
Extract:
- tar -zxvf arelle-redhat-x86_64-2014-12-31.tar.gz
Move into directory:
- cd arelle-redhat-x86_64-2014-12-31
run the following to verify & see help:
- ./arelleCmdLine -h
Usage: arelleCmdLine [options]
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-f ENTRYPOINTFILE, --file=ENTRYPOINTFILE
FILENAME is an entry point, which may be an XBRL
instance, schema, linkbase file, inline XBRL instance,
testcase file, testcase index file. FILENAME may be a
local file or a URI to a web located file.
--username=USERNAME user name if needed (with password) for web file
retrieval
--password=PASSWORD password if needed (with user name) for web retrieval
-i IMPORTFILES, --import=IMPORTFILES
FILENAME is a list of files to import to the DTS, such
as additional formula or label linkbases. Multiple
file names are separated by a '|' character.
-d DIFFFILE, --diff=DIFFFILE
FILENAME is a second entry point when comparing
(diffing) two DTSes producing a versioning report.
-r VERSREPORTFILE, --report=VERSREPORTFILE
FILENAME is the filename to save as the versioning
report.
-v, --validate Validate the file according to the entry file type.
If an XBRL file, it is validated according to XBRL
validation 2.1, calculation linkbase validation if
either --calcDecimals or --calcPrecision are
specified, and SEC EDGAR Filing Manual (if --efm
selected) or Global Filer Manual disclosure system
validation (if --gfm=XXX selected). If a test suite or
testcase, the test case variations are individually so
validated. If formulae are present they will be
validated and run unless --formula=none is specified.
--calcDecimals Specify calculation linkbase validation inferring
decimals.
--calcPrecision Specify calculation linkbase validation inferring
precision.
--efm Select Edgar Filer Manual (U.S. SEC) disclosure system
validation (strict).
--disclosureSystem=DISCLOSURESYSTEMNAME
Specify a disclosure system name and select disclosure
system validation. Enter --disclosureSystem=help for
list of names or help-verbose for list of names and
descriptions.
--hmrc Select U.K. HMRC disclosure system validation.
--utr Select validation with respect to Unit Type Registry.
--utrUrl=UTRURL Override disclosure systems Unit Type Registry
location (URL or file path).
--infoset Select validation with respect testcase infosets.
--labelLang=LABELLANG
Language for labels in following file options
(override system settings)
--labelRole=LABELROLE
Label role for labels in following file options
(instead of standard label)
--DTS=DTSFILE, --csvDTS=DTSFILE
Write DTS tree into FILE (may be .csv or .html)
--facts=FACTSFILE, --csvFacts=FACTSFILE
Write fact list into FILE
--factListCols=FACTLISTCOLS
Columns for fact list file
--factTable=FACTTABLEFILE, --csvFactTable=FACTTABLEFILE
Write fact table into FILE
--concepts=CONCEPTSFILE, --csvConcepts=CONCEPTSFILE
Write concepts into FILE
--pre=PREFILE, --csvPre=PREFILE
Write presentation linkbase into FILE
--cal=CALFILE, --csvCal=CALFILE
Write calculation linkbase into FILE
--dim=DIMFILE, --csvDim=DIMFILE
Write dimensions (of definition) linkbase into FILE
--formulae=FORMULAEFILE, --htmlFormulae=FORMULAEFILE
Write formulae linkbase into FILE
--viewArcrole=VIEWARCROLE
Write linkbase relationships for viewArcrole into
viewFile
--viewFile=VIEWFILE Write linkbase relationships for viewArcrole into
viewFile
--roleTypes=ROLETYPESFILE
Write defined role types into FILE
--arcroleTypes=ARCROLETYPESFILE
Write defined arcrole types into FILE
--testReport=TESTREPORT, --csvTestReport=TESTREPORT
Write test report of validation (of test cases) into
FILE
--testReportCols=TESTREPORTCOLS
Columns for test report file
--rssReport=RSSREPORT
Write RSS report into FILE
--rssReportCols=RSSREPORTCOLS
Columns for RSS report file
--skipDTS Skip DTS activities (loading, discovery, validation),
useful when an instance needs only to be parsed.
--skipLoading=SKIPLOADING
Skip loading discovered or schemaLocated files
matching pattern (unix-style file name patterns
separated by '|'), useful when not all linkbases are
needed.
--logFile=LOGFILE Write log messages into file, otherwise they go to
standard output. If file ends in .xml it is xml-
formatted, otherwise it is text.
--logFormat=LOGFORMAT
Logging format for messages capture, otherwise default
is "[%(messageCode)s] %(message)s - %(file)s".
--logLevel=LOGLEVEL Minimum level for messages capture, otherwise the
message is ignored. Current order of levels are
debug, info, info-semantic, warning, warning-semantic,
warning, assertion-satisfied, inconsistency, error-
semantic, assertion-not-satisfied, and error.
--logLevelFilter=LOGLEVELFILTER
Regular expression filter for logLevel. (E.g., to not
match *-semantic levels,
logLevelFilter=(?!^.*-semantic$)(.+).
--logCodeFilter=LOGCODEFILTER
Regular expression filter for log message code.
--parameters=PARAMETERS
Specify parameters for formula and validation
(name=value[,name=value]).
--parameterSeparator=PARAMETERSEPARATOR
Specify parameters separator string (if other than
comma).
--formula=FORMULAACTION
Specify formula action: validate - validate only,
without running, run - validate and run, or none -
prevent formula validation or running when also
specifying -v or --validate. if this option is not
specified, -v or --validate will validate and run
formulas if present
--formulaParamExprResult
Specify formula tracing.
--formulaParamInputValue
Specify formula tracing.
--formulaCallExprSource
Specify formula tracing.
--formulaCallExprCode
Specify formula tracing.
--formulaCallExprEval
Specify formula tracing.
--formulaCallExprResult
Specify formula tracing.
--formulaVarSetExprEval
Specify formula tracing.
--formulaVarSetExprResult
Specify formula tracing.
--formulaVarSetTiming
Specify showing times of variable set evaluation.
--formulaAsserResultCounts
Specify formula tracing.
--formulaSatisfiedAsser
Specify formula tracing.
--formulaUnsatisfiedAsser
Specify formula tracing.
--formulaUnsatisfiedAsserError
Specify formula tracing.
--formulaFormulaRules
Specify formula tracing.
--formulaVarsOrder Specify formula tracing.
--formulaVarExpressionSource
Specify formula tracing.
--formulaVarExpressionCode
Specify formula tracing.
--formulaVarExpressionEvaluation
Specify formula tracing.
--formulaVarExpressionResult
Specify formula tracing.
--formulaVarFilterWinnowing
Specify formula tracing.
--formulaVarFiltersResult
Specify formula tracing.
--formulaRunIDs=FORMULARUNIDS
Specify formula/assertion IDs to run, separated by a
'|' character.
--uiLang=UILANG Language for user interface (override system settings,
such as program messages). Does not save setting.
--proxy=PROXY Modify and re-save proxy settings configuration.
Enter 'system' to use system proxy setting, 'none' to
use no proxy, 'http://[user[:password]@]host[:port]'
(e.g., http://192.168.1.253, http://example.com:8080,
http://joe:secret@example.com:8080), or 'show' to
show current setting, .
--internetConnectivity=INTERNETCONNECTIVITY
Specify internet connectivity: online or offline
--internetTimeout=INTERNETTIMEOUT
Specify internet connection timeout in seconds (0
means unlimited).
--internetRecheck=INTERNETRECHECK
Specify rechecking cache files (weekly is default)
--internetLogDownloads
Log info message for downloads to web cache.
--xdgConfigHome=XDGCONFIGHOME
Specify non-standard location for configuration and
cache files (overrides environment parameter
XDG_CONFIG_HOME).
--plugins=PLUGINS Modify plug-in configuration. Re-save unless 'temp'
is in the module list. Enter 'show' to show current
plug-in configuration. Commands show, and module urls
are '|' separated: +url to add plug-in by its url or
filename, ~name to reload a plug-in by its name, -name
to remove a plug-in by its name, relative URLs are
relative to installation plug-in directory, (e.g.,
'+http://arelle.org/files/hello_web.py', '+C:\Program
Files\Arelle\examples\plugin\hello_dolly.py' to load,
or +../examples/plugin/hello_dolly.py for relative use
of examples directory, ~Hello Dolly to reload, -Hello
Dolly to remove). If + is omitted from .py file
nothing is saved (same as temp). Packaged plug-in
urls are their directory's url.
--packages=PACKAGES Modify taxonomy packages configuration. Re-save
unless 'temp' is in the module list. Enter 'show' to
show current packages configuration. Commands show,
and module urls are '|' separated: +url to add package
by its url or filename, ~name to reload package by its
name, -name to remove a package by its name, URLs are
full absolute paths. If + is omitted from package
file nothing is saved (same as temp).
--packageManifestName=PACKAGEMANIFESTNAME
Provide non-standard archive manifest file name
pattern (e.g., *taxonomyPackage.xml). Uses unix file
name pattern matching. Multiple manifest files are
supported in archive (such as oasis catalogs).
(Replaces search for either .taxonomyPackage.xml or
catalog.xml).
--abortOnMajorError Abort process on major error, such as when load is
unable to find an entry or discovered file.
--showEnvironment Show Arelle's config and cache directory and host OS
environment parameters.
--collectProfileStats
Collect profile statistics, such as timing of
validation activities and formulae.
--webserver=WEBSERVER
start web server on host:port[:server] for REST and
web access, e.g., --webserver locahost:8080, or
specify nondefault a server name, such as cherrypy,
--webserver locahost:8080:cherrypy. (It is possible to
specify options to be defaults for the web server,
such as disclosureSystem and validations, but not
including file names.)
--store-to-XBRL-DB=STOREINTOXBRLDB
Store into XBRL DB. Provides connection string: host,
port,user,password,database[,timeout[,{postgres|rexste
r|rdfDB}]]. Autodetects database type unless 7th
parameter is provided.
--load-from-XBRL-DB=LOADFROMXBRLDB
Load from XBRL DB. Provides connection string: host,p
ort,user,password,database[,timeout[,{postgres|rexster
|rdfDB}]]. Specifies DB parameters to load and
optional file to save XBRL into.
-a, --about Show product version, copyright, and license.
[ make sure you have installed python and pg8000 ] run:
- pip install pg8000
Creating the database:
Download database from here: https://github.com/Arelle/Arelle/blob/master/arelle/plugin/xbrlDB/xbrlSemanticPostgresDB.ddl:
- wget --no-check-certificate https://github.com/Arelle/Arelle/blob/master/arelle/plugin/xbrlDB/xbrlSemanticPostgresDB.ddl
- su - postgres
- create database sec
- psql -h HOST -U USERNAME -d sec -a -f xbrlSemanticPostgresDB.ddl
Add plugins:
see all installed plugins:
- ./arelleCmdLine --plugins show
[info] Plug-in modules: -
[info] Plug-in: XBRL Database; author: Mark V Systems Limited; version: 0.9; status: enabled; date: 2014-12-09T04:41:53 UTC; description: This plug-in implements the XBRL Public Postgres, Abstract Model and DPM Databases. ; license Apache-2 (Arelle plug-in), BSD license (pg8000 library). - xbrlDB
Install plugin xbrlDB:
- ./arelleCmdLine --plugins +xbrlDB
[info] Addition of plug-in XBRL Database successful. - xbrlDB
Scrape, parse and populate SEC filing data into DB:
run:
- ./arelleCmdLine -f https://www.sec.gov/Archives/edgar/xbrlrss.all.xml -v --store-to-XBRL-DB "HOST,5432,usrname,password,sec,120,pgSemantic"
This will download latest 100 SEC filings and store tag based xbrl data at database HOST:5432/sec
To download,parse and store all SEC filing for a single month say 2016 January we can simple run:
- ./arelleCmdLine -f https://www.sec.gov/Archives/edgar/monthly/xbrlrss-2016-01.xml
- -v --store-to-XBRL-DB "HOST,5432,username,password,sec,120,pgSemantic"
Say for a single company central index key 0001372183 the following query will get all tag based financial information:
- select distinct period.end_date from aspect,data_point,period,entity_identifier where aspect.aspect_id=data_point.aspect_id and data_point.report_id=entity_identifier.report_id and period.period_id=data_point.period_id and entity_identifier.identifier='0001372183';
dei_AmendmentFlag,false,2015-12-01
dei_CurrentFiscalYearEndDate,--02-29,2015-12-01
dei_DocumentFiscalPeriodFocus,Q3,2015-12-01
dei_DocumentFiscalYearFocus,2016,2015-12-01
dei_DocumentPeriodEndDate,2015-11-30,2015-12-01
dei_DocumentType,10-Q,2015-12-01
dei_EntityCentralIndexKey,0001372183,2015-12-01
dei_EntityCommonStockSharesOutstanding,5491753,2016-01-19
dei_EntityFilerCategory,Smaller Reporting Company,2015-12-01
dei_EntityRegistrantName,"Monaker Group, Inc.",2015-12-01
dei_TradingSymbol,MKGI,2015-12-01
invest_InvestmentWarrantsExercisePrice,0.05,2012-08-22
invest_InvestmentWarrantsExercisePrice,3,2009-03-01
mkgi_AdditionalExpendituresForCostsAssociatedWithEmploymentWebsite,10000,2015-12-01
mkgi_AdditionalOffsettingRentExpenseMonthly,2500,2015-12-01
mkgi_AdvancesConversionConvertedIntoPromissoryNoteAmount,70000,2011-04-14
mkgi_AdvancesToFormerSubsidiary,75000,2015-12-01
mkgi_AssetsImpairmentChargesShares,0,2015-12-01
mkgi_AssignmentOfPrincipalToNonRelatedParty,225000,2012-02-16
mkgi_CarryingValueOfBusinessAfterAdjustments,7811286,2014-11-01
mkgi_CarryingValueOfBusinessAfterAudit,1556098,2014-11-01
mkgi_CashFromMerger,0,2014-12-01
mkgi_CashFromMerger,56902,2015-12-01
...............................................................
...............................................................
...............................................................
Full Output File:
https://www.dropbox.com/s/m9finc2x88k75jl/sec-filing-1372183.csv?dl=0
Here is a list of all us-gaap tags with descriptions:
https://github.com/ifanchu/pyXBRL/blob/master/us-gaap/concepts_2014.csv
Cleaning up:
By default arelle puts every downloaded filings at:
/root/.config/arelle/cache/
Sp periodically you may need cleaning up:
- rm -rf /root/.config/arelle/cache/*
References:
http://arelle.org/documentation/xbrl-database/
http://www.openfiling.info/wp-content/upLoads/data/ArelleUsersManual.pdf
http://arelle.org/wordpress/wp-content/uploads/downloads/2011/09/ComparabilityAndDataMiningUnifiedModel-Paper.pdf
Related:
http://mohiplanet.blogspot.com/2016/02/installing-arelle-32-bit-on-windows-7.html
No comments:
Post a Comment