YLoader User Manual
Command Line Options

This section describes the command line usage for both the classic GUI, and the new console versions of the YLoader application.

Most command line arguments have corresponding settings in the GUI version of YLoader. The ones that do not, are generally used to control the runtime behavior of the console version, without affecting the downloaded data.

Both the GUI and console versions accept the same command line arguments with some differences as far as the default values for missing arguments, or required vs. optional arguments:

  • for the GUI version, if an argument is not defined on the command line, it will use the corresponding value most recently set in either the main application window or Settings dialog. The console version has fixed defaults which are used in case the corresponding command line argument is not set (for optional arguments).
  • for the GUI version all command line arguments are optional. The console version requires the symbols list file, output path and the datasource to be specified on the command line
  • each of the two versions have specific settings that are ignored by the other, generally settings intended to control the application. For example, autostart and autoexit arguments are used by the GUI version to allow it to run in unattended mode, but are ignored by the console version which runs in unattended mode by default.

The console version will display all activity in the console window: current settings, session statistics, session progress (as a horizontal progress bar and percentage), each symbol download status. In addition, the console version can also save all activity to a log file, containing also current settings and all downloading activity (or optionally only errors).

Passing arguments

There are two nonexclusive ways to pass startup arguments:

  • directly on the command line
  • in one or more configuration files.

Passing arguments on the command line

The command line has the general format:

<executable_name> [-argument1 value1] [-argument2 value2] [-argument3 value3]...

where <executable_name> is:

  • yloader.exe for the GUI version
  • ylodercl.exe for the console version.

Each argument is specified using a name - either short: one letter preceded by "-" (ex: -a), or long: a string preceded by "--" (ex: --autostart).

It is not required that the entire long name be entered, only enough characters to uniquely identify the argument. For example --auto will trigger an error because the application doesn't know if that refers to --autoexit or --autostart. But --autos will be accepted as it is unambiguously referring to --autostart.

Most arguments require a value following the argument name. The argument name and value must be separated by one or more spaces or tabs. The value can can be optionally surrounded by quotes if it must contain spaces (for example a path or file name). Some arguments do not require a value, and they can be followed immediately by another argument name.

There are several types of values required by the various arguments:

  • path, for example -s: or --symbolsfile. Can be any legal path, optionally surrounded by quotes, which is necessary if it contains spaces. The character "\" should be used to indicate different directories levels. Ex: "c:\my symbols\nasdaq\all_nasdaq.txt"
  • boolean, for example -u or --update. A boolean value can be true or false. True can be represented by the string "true" (or any case combination: True, tRuE, etc, with or without surrounding quotes) or by the number 1. False can be represented by the string "false" (again using any combination of lower and upper case: False, FALSe, etc., with or without surrounding quotes) or by the number 0. Any other values will trigger an error.
  • string, for example -h or --datafileheader: A string must contain at least one character. An empty string surrounded by quotes ("") will be interpreted as a missing value and will trigger an error. To include space or tab characters surround the string by quotes. A few special characters can be represented using an escape sequence:
    • new line: \n
    • tab: \t
    • quote: \"
    • backslash: \\

    This allows more complex strings to be entered despite the the command line or configuration file limitations.

    For example a data file header on two lines can be specified as: "data file header line1\ndata file header line2"

  • integer, for example -c or --connections. This is represented as a series of decimal digits: ex: -c 15. Any other character will trigger an error.
  • floating point, for example -M or --volumemultiplier. This is represented by any valid floating point value: 15, 2e2, 15.324534 etc.

For the GUI version, if an argument is not present on the command line, the value of the corresponding option will be loaded from the previously saved values. The console version will use preset defaults if these arguments are not present on the command line (except for the required ones).

Passing arguments in a configuration file

Configuration files can be a more convenient way to pass arguments to the application than directly on the command line. For example, if the application requires many command line parameters with long arguments, the command string can become very long and difficult to read and maintain. Or if the YLoader arguments are auto generated as part of a larger script run in unattended mode.

To specify a configuration file use the command line argument -C or --configfile followed by the configuration file name. This argument should only be entered on the command line, although if it is present in the configuration file itself, it will simply be ignored. This is to avoid the case of an infinite loop, which would happen if a configuration file could directly or indirectly load itself.

Multiple configuration files can be specified:

-C "config_file1" -C "config_file2"

The configuration file format is:

argument_name1=value1
argument_name2=value2
argument_name3
#comment1

#comment2
...

As shown above, a configuration file can contain any number of argument_name=value pairs, with at most one argument per line. Only argument long names without the preceding "--" are allowed in the configuration file (see the supported arguments table below for a description of long names). Abbreviated argument names are accepted as long as they are not ambiguous (ex: con which will be interepreted as connections)

Arguments that normally do not require a value (such as autostart, autoexit, nodateseparator etc.) can also be set in the configuration file, in which case only the argument name will be present on the line. Ex:

autoexit
autostart

All argument name and value rules that apply to arguments passed on the command line, also apply to arguments passed in configuraiton files. For example, values can be surrounded by quotes or not, string values can contain escaped characters, etc.

Lines that start with #, which are generally used for descriptinve comments, as well as empty lines, are allowed in configuration files, and ignored by YLoader

Passing arguments in a configuration files has several advantages over passing them on the command line:

  • configuration files are visually cleaner and clearer than the corresponding command line./li>
  • comments can be used to enhance clarity
  • can contain arguments that are frequently used so they won't have to be re-entered each time
  • can be easily generated by external scripts in case of automated runs.

This being said, the command line is generally more succinct, as it allows short argument names. It is also more convenient when just a few arguments need to be specified.

Users are encouraged to experiment with both methods of passing arguments, and even combine them for the right balance required by the specific application.

Supported arguments

Note that all arguments names, short and long, are case sensitive - for example -s sets the symbols file, while -S sets the data sort order within a data file.

Short name Long name Value GUI Console Description Example
Application control
-? --help   optional optional Displays a short description of all the command line options yloader -?
-a --autostart   optional ignored If present, this argument will automatically start a downloading session yloader -a
-x --autoexit   optional ignored Instructs the downloader to automatically close after the downloading session has completed. It can be used together with argument -a to automatically start and end a YLoader session.

Note: options -a and -x are generally used when YLoader is used as part of an automated script. In that case, it makes more sense to use the console version of YLoader instead of the GUI version.

yloader -a -x
-o --showsettings   ignored optional (default - don't show settings) Available only in the console version. Instructs the downloader to display all settings for the session. Used to see the actual values for all the settings, even those that have not been set explicitly on the command line. yloadercl -o
-N --dontrun   ignored optional
(default - run)
Available only in the console version. Instructs the downloader not to run a downloading session. Can be used together with -o to check the values of various settings before actually starting a session. yloadercl -o -N
-R --progress N (integer) ignored optional
(default - 100)
Available only in the console version. Instructs the downloader to display progress information every N downloaded symbol if N > 0, or no progress information if set to 0  
-C --configfile a file name optional optional
(default - empty string)
Instructs the downloader to load arguments from a configuration file in addition to those passed directly on the command line. yloader -C "\yloader.cfg"
Download session control
-s --symbolsfile a string containing the symbols file name optional required Sets the name and path of the file containing the symbols. The value string must be between quotes if it contains spaces yloader -s "c:\symbols\nasdaq.txt"
-d --datapath a string containing the data path optional required Sets the path under which the generated files will be saved. The value string must be between quotes if it contains spaces yloader -d "c:\data"
-n --datasource a string identifying the datasource optional required Instructs the downloader which datasource to use to download the data. The datasource name can be incomplete as long as it uniquely identifies it, for example "y", "ya" "go" are all valid datasource names. Also, the name is case insensitive, so both "y" and "Y" will identify Yahoo as datasource. yloader -n "Y"
yloader -n "yahoo"
yloader -n "goo" )
yloader -n "4389DD"

Note: the last example will set Google as plugin based on its id.

-j --adjust 0 to disable
1 to enable
optional optional
(default on)
Instructs the downloader whether to adjust data for split and dividends or not. Note that some data sources may not support either adjusted or unadjusted data. yloader -j 1
-v --validateprices 0 to disable
1 to enable
optional optional
(default on)
Instructs the downloader whether to check data for errors or not. yloader -v 0
-t --startdate 0 to disable
1 to enable
optional optional
(default - earliest date available)
Sets the start date of the downloading range. The format is "mm/dd/yyyy" where mm is the month (1-12), dd is the day (1-31) and yyyy is the year. An empty quoted string can be set to specify the earliest available start date. See also -l and -u. yloader -t "1/1/2008"
yloader -t ""
-e --enddate 0 to disable
1 to enable
optional optional
(default - most recent date available)
Sets the end date of the downloading range. The format is "mm/dd/yyyy" where mm is the month (1-12), dd is the day (1-31) and yyyy is the year. An empty quoted string can be set to most recent available end date. See also -l and -u. yloader -e "1/1/2008"
yloader -e ""
-l --allavailable 0 to disable
1 to enable
optional optional
(default - off)
Instructs the downloader whether to ignore the start/end date settings and download all available data ,if enabled, or use the start/end dates if disabled. yloader -l 1
-u --update 0 to disable
1 to enable
optional optional
(default - off)
Instructs the downloader to download only the most recent missing bars if enabled, or the data for the entire specified range if disabled. Enabling the update mode is the most efficient way to download data on an ongoing basis, as this will always download the minimum amount of data.

Note: even in update mode the downloader may decide to download all data (according to the specified range) if it determines that old data is no longer consistent with the new data due to adjustments. See also argument -w

yloader -u 0
-p --period 0 for daily
1 for weekly
2 for monthly
optional optional
(default - daily)
Set the data resolution yloader -p 1
-c --connections number between 1-20 optional optional
(default - 5)
Sets the number of simultaneous downloading connections. If bandwidth and processing power were unlimited, the higher the number the faster the download. In practice, should be used with caution, as a higher number may not necessarily result in a faster download yloader -c 7
-i --invaliddatahandling 0 - warning
1 - error
optional optional
(default - warning)
Instructs the downloader to issue an warning and continue downloading the data, if set to warning, or issue an error and halt downloading the data for that symbol, if set to error. yloader -i 1
-V --volume0handling 0 - handle as valid
1 - handle as invalid
optional optional
(default - valid)
Instructs the downloader how to handle the case of volume 0. If set to valid, it will consider volume 0 as a valid value, if set to invalid, it will consider volume 0 as invalid, and it will handle further according to argument -i (handling of invalid data) yloader -V 0
-w --noreloadolddata 0 - allow reloading of old data
1 - don't allow reloading of old data
optional optional
(default - 0)
Instructs the downloader to reload old data in update mode, if necessary, if set to enable, and disables reloading of old data in update mode, even if old data is inconsistent, if disabled. See also argument -u. yloader -w 1
Settings based line formatting
-f --dateformat 0 for m/d/y
1 for d/m/y
2 for y/m/d
optional optional
(default - 0)
Sets the date format for the downloaded data. yloader -f 1
-y --addsymbol 0 to disable
1 to enable
optional optional
(default - off)
Instructs the downloader whether to add the symbol to the generated file if enabled (see argument -m for the column where the symbol will be placed), or to not add the symbol to the generated file if disabled. yloader -y 1
-m --symbolcolumn number between 0-6 optional optional
(default 0)
If adding the symbol to the generated file is enabled (see argument -y), this will set the column in which the symbol will be added (0 for the first column, 1 for the second, etc, up to 6 for the last column) yloader -y 1 -m 2
-r --dateseparator a string of 1 or more characters optional optional
(default - "/")
Sets the date separator to the specified character or string. To set the "no date separator" mode, use command line argument -z. The separator string should be surrounded by quotes in case it starts with "-", which can cause an ambiguous command line. If both -r and -z are present on the command line, -z takes precendence, so there won't be a date separator yloader -r "-"
-z --nodateseparator   optional optional Sets the "no date separator" mode. See command line argument -r for how to set any date separator. If both -r and -z are present on the command line, -z takes precedence, so there won't be a date separator yloader -z
-F --fieldseparator a string of 1 or more characters optional optional
(default - ",")
Sets the field separator to the specified character or string yloader -F "+"
-D --paddatefields 0 to disable
1 to enable
optional optional
(default - disable)
Instructs the downloader whether to pad the date fields with 0s if enabled, or to leave the date unchanged if disabled. The padding will only affect the day and month fields, if either of these only has one digit (ex: 1/1/2010 becomes 01/01/2010) yloader -D 1
-M --volumemultiplier positive float optional optional
(default - 1)
Sets the volume multiplier. yloader -M 0.1
-L -fixeddecimalscount integer optional optional (default -1) Saves price values with fixed number of decimals set by the argument value. If the value has fewer decimal digits, value will be padded with zeros up to the specified number of digits. Use "-1" for default behavior, which is variable number of decimals yloader -L 2
-U -decimalseparator a string of one or more characters optional optional (default ".") Sets the decimal separator character. The default is "." yloader -U ","
Regex based line formatting
-O --enableregexformatting 0 to disable
1 to enable
optional optional (default - disable) it will enable regular expression formatting of output data and will disable settings based output data formatting. All other output data formatting arguments (such as field separator, date format etc) will be ignored.  
-Y --matchregex a string (the regular expression) optional optional
(default - regex to extract all fields)
If set, it will define the regular expression used to extract fields from the input data.
See the chapter on regular expression formatting on more information
yloader -O 1 -Y "(.*),(.*)"
-Z --formatstring a string describing the data format ignored optional
(default - it will leave input data unchanged, if based on the default match regex)
If set, it will define the format of each line in the output data files. See the chapter on regular expression formatting on more information yloader -O 1 -Z "$1,$2
-G --regexflags an integer ignored optional
(default 0)
This argument is currently not used. In the future it will allow fine tuning of the regular expression formatting (the type of expression, character case handling etc). See the chapter on regular expression formatting on more information yloader -O 1 -G 0
Output files
-S --sortascending 0 for descending
1 for ascending
optional optional
(default - ascending)
Sets the bar sorting order in the generated files relative to their date. yloader -S 0
-h --datafileheader a string of characters optional optional
(default - empty string)
Instructs the downloader to add the string to the top of all data files. If the string contains spaces, it should be quoted. Currently this string cannot contain special characters such as new line etc. To enable a multi-line header string, use the Settings/Formatting/Data File Header when using the GUI version of the downloader. yloader -h "This is a header string"
-P --prependtofilename a string of characters optional optional
(default - empty string)
Instructs the downloader to add the specified string to the beginning of the file name for each symbol. If the strings contains spaces, it should be quoted. The string cannot contain characters that are invalid in file names, such as ?, *, " yloader -P "data"
-A --appendtofilename a string of characters optional optional
(default - empty string)
Instructs the downloader to add the specified string to the end of the file name (before the extension) for each symbol. If the strings contains spaces, it should be quoted. The string cannot contain characters that are invalid in file names, such as ?, *, " yloader -A "data"
-X --extension a string of characters optional optional
(default - "csv")
Instructs the downloader to use the specified string as the file extension. If the strings contains spaces, it should be quoted. The string cannot contain characters that are invalid in file names, such as ?, *, " yloader -X "txt"
-b --singleoutputfile a file name optional optional
(default - empty string)
Instructs the downloader to save all the downloaded data to a single file with the specified name. Files for individual symbols will still be generated. If the strings contains spaces, it should be quoted. The string cannot contain characters that are invalid in file names, such as ?, *, " yloader -b "c:\allsymbolsfile.csv"
-k --nosingleoutputfile   optional not used Instructs the downloader to clear the single output file name field, thus disabling the single output file generation feature for the current session (used only in the GUI version). yloader -k
-q --createsubdirs 0 - disable
1 - enable
optional optional
(default - 0)
Instructs the downloader to save the data files under a hierarchy of nested folders (using the first two characters of each symbol) if enabled, or to save the output files in a flat structure, if disabled. The output data path set with -d is used as root in both cases. yloader -q 1
-K
--mapcharacter
a string of one or two characters
optional
optional
Maps one character in the symbol name to one character in the data file for that symbol. There can be up to four mappings. The main purpose of this argument is to allow saving data for symbols that contain characters that are invalid in file names, such as ":" etc.
  • If the string contains one character, it will be removed from the data file name, e.g. for symbol "ry:ca" and mapping ":", the data file name will be "ryca" (the ":" will be removed).
  • If the string contains two characters, the first character in the symbol name will be mapped to the second character in the data file name. e.g. for symbol "ry:ca" and mapping ":_", the data file name will be "ry_ca".
yloader -K ":_"
yloader -K ":"
Extra Symbols Lists
-B --errorsymbolsfilename error symbols list file name optional optional
(default - no error symbols file name)
Sets the error symbols list file name. If set, it will save all symbols with errors during the current session into this file, otherwise will not save the these symbols. Can be used together with arguments -H and -I for more control over how to save and use the error symbols file. yloader -B ""\errors.txt""
-H --appendtoerrorsymbolslist 0 - do not append to error symbols file
1 - append to error symbols file
optional optional
(default - 1
Instructs the downloader to append new error symbols to the existing error symbols file (if any), if set to 1, or to create a new error symbols file after each session if set to 0. Can be used together with arguments -B and -I for more control over how to save and use the error symbols file. yloader -H "0"
-I --ignoreerrorsymbolslist 0 - do not ignore symbols in error symbols list
1 - ignore symbols in error symbols list
optional optional
(default - 1
Instructs the downloader to ignore symbols that appear in the error symbols list file (if any), during a downloading session, , even if they appear in the main list of symbols to download, if set to 1, or to not ignore these symbols if set to 0. Can be used together with arguments -B and -H for more control over how to save and use the error symbols file. yloader -I "1"
-J --ignoresymbolslist file name containing symbols to be ignored
no file name
optional optional
(default - 1
If set to the name of a file containing a list of symbols, these symbols will be ignored during a downloading session even if they appear in the main list of symbols to download. This file can be used in addition to the automatically generated error symbols list file to further control what is and what is not downloaded. yloader -J "\exclude_symbols.txt"
Log file
-g --logfile a file name optional optional
(default - no log file name)
Sets the the log file name. If set, it will save the current session activity log with this name, otherwise will not save the activity log. Can be used together with argument -z for more control over how to save the log file. yloader -g "\yloader.log"
-E --logonlyerrors 0 - log all events
1 - log only error events
optional optional
(default - log everything)
Instructs the downloader to only log error events during a downloading session yloader -E 1
-T --appendtolog 0 - reset the log file
1 - append to the log file
optional optional
(default - 0)
Instructs the downloader to append log statements for new sessions to the existing log file if set to 1, or to reset the log file every time a new session starts. yloader -z 1

Examples

1. autostart and/or autoexit YLoader

yloader.exe --autoexitt
yloader.exe -x
yloader.exe -x -a

2. Set the required arguments for the console version, using long argument names

yloadercl.exe --symbolsfile "c:\symbols files\nasdaq.txt" --datapath "c:\data" --datasource "yahoo"

3. Set several options (adjust off, all available off, update on, validate prices off, data source Google)and auto-start a downloading session.

yloader.exe -j 0 -l 0 -u 1 -v 0 -o 1 -a

4. Set several options in a configuration file

yloadercl.exe -C "\yloaderconfig.txt"
where the configuration file "yloaderconfig.txt" contains:
symbolsfile="c:\symbols files\nasdaq.txt"
datapath="c:\data"
datasource="yahoo"

5. Display the current settings, including the defaults for missing arguments and exit (console version only).

yloadercl.exe -o -N <required arguments>

Note: -o will display current settings, -N will stop the application before the downloading stats. <required arguments> indicates that arguments that are needed to run the application (such as data path, symbols file, data source) must still be passed or an error will be triggered before the value of all arguments is displayed.

6. Run a console session for data within a defined range

yloadercl -d \data -s "\symbols.txt" -n "yahoo" -t 7/7/2010 -e 10/25/2010

7. Run a console session in update mode (download only most recent bars if possible)

yloadercl -d \data -s "\symbols.txt" -n "y" -u 1

Note: -n "y" will instruct YLoader to use Yahoo as data source (a few characters of the datasource name are enough to identify it)