ComCap
Release Notes

1 - The Unlimited version now supports capture from up to 500 serial COM ports, TCP/IP Server, TCP/IP Client and UDP/IP network protocols simultaneously, with suitable hardware. It has been tested on Windows XP with 260 TCP/IP sessions created by ComGen (which never had a limit), writing capture logs one line per second, with only 2-6% CPU usage on a old Pentium 4 and 12 megs memory usage.

2 - ComCap, ComGen and Signals now all support serial COM ports with names other than COM followed by a number, ie CNCA2. Virtual COM ports are sometimes installed with such strange names.

3 - When capturing data on the same channel from multiple UDP clients using different IP addresses, the correct IP of the last packet is now displayed and added to the capture log if the Remote IP Address is added as escaped text (\r), rather than the IP of the first packet only.

4 - It's now possible to use keyboard short cuts to copy text from the capture log windows, as well as the right click menu options.

5 - Improved the logging for license key failures, particularly when old version 3 keys are used by accident with ComCap version 4 (which needs newer keys).

2 - In Capture Settings, General, a 'Line End Timeout' may now be specified in seconds, where zero means no timeout, up to 300 seconds. When the timeout expires, an incomplete captured line will be processed, saved and displayed.  This is usually only necessary when non-ASCII data is being captured where there are no carriage returns or line feeds, but may also be useful when setting up ComCap to capture from a serial port with unknown speed, since it can be used to cause display of the 'corrupted' data caused by speed mismatch which will be missing line ends.

3 - In Capture Settings, Logging, 'Ignore Blank Lines' causes lines without any printable characters to be ignored.  This will reduce the size of logs that contain far too many blank lines.  This option should generally be used with 'Remove Control Characters'.

4 - In Capture Settings, Logging, 'Log Hex Data' causes all captured data to be converted into hexadecimal (doubling the size).  This is primarily intended for capturing binary data, but may be used a debugging tool to find the exact format of data being captured, for instance the type of line endings.  Normal line ending rules are applied when capturing in hex, so lines may be broken on linefeed, etc (also saved in hex).

5 - The actual information log file names are now logged hourly.  This may be useful where an emergency log is opened due to an access conflict on the original file.

6 - When configuring ComCap to 'Capture Using Background Service', a warning is now displayed if any channels are configured as 'Save to  Database' and a service account has not been specified, since database access needs logon credentials.

7 - It's now again possible to specify UNC network paths in Log Directories with 'Capture Using Background Service'. 

8 - When starting the ComCap background service from the ComCap tray version, the channel currently selected now starts to update immediately all channels have started, rather than the next time it's clicked. 

9 - When using the TCP Server protocol to Echo captured Data, or Send Remote the Information Log or Alerts, up to five remote TCP Clients can now access the TCP Server simultaneously (previously just one). This allows ComCap to be used to distribute captured data to five other PCs, perhaps also running ComCap. 

10 - The highest numbered COM port recognised by ComCap has been increased from COM60 to COM200 to allow for virtual COM ports being installed with high numbers.  Also, the Standard version now lists all installed COM ports, not just the first three, but still only allows capture from a maximum of three ports.

11 - If the requirement is to capture serial data from another application on the same PC, ComCap now includes a Null Modem Emulator (com0com) from http://com0com.sourceforge.net/ that installs a linked pair of virtual serial ports, instead of needing to use a physical pair of COM ports and a null modem cable.

15 - Made a few improvements in ComGen. It will now close down faster when Windows itself closes down. UDP Client now pings the remote host before sending any data to establish it exists, and continues to ping during the session to ensure it's still available. Corrected the default Data Types that still had \s8 for a serial number (from an earlier beta) instead of \#8 used in the final release.

ComCap v4 is effectively a new application, and may be installed in parallel with ComCap v3, although only one will be able to access the same serial ports at the time. ComCap v3 license keys will run for 24 hours in v4, for trial purposes. All settings need to be newly created for a first install of ComCap v4, please see below for information on how settings have changed. Also note that there is no longer a separate service version of  ComCap, the service version is now effectively integrated into the tray application, and is now automatically installed or de-installed, and enabled, through Preferences, and controlled by the tray application.

1 - Network Capture Functionality

ComCap now has the ability to capture network data, using TCP/IP and UDP/IP protocols. Network protocols are more flexible than serial communications, with potentially infinite distances and speed. Many network appliances output log information, typically using the 'syslog' protocol, and as telephone switches become network aware (VoIP) they are  offering network logging instead of the serial port. There are also serial device servers that 'convert' serial port data to network protocols, from Lantronix, RE Smith and Tysso eCov for instance, easing data capture distances.

Network capture is initially set-up on the Common Settings Network tab. A new blank row will have been created for a network channel which should be given a Capture Name. The Protocol should be selected from the drop down box, as UDP Server, TCP Client or TCP Server. The Service defaults to Stream so ComCap just captures the data, but may be changed to Syslog or SNMP using the drop down box if those protocols are being captured. The Local IP Address is also from a drop down box, 0.0.0.0 means the computer will listen on all IP addresses allocated to the computer, if more than one, otherwise a specific IP may be selected. Local IP Port is only needed for UDP and TCP Server and needs to be the port on which the remote PC will be sending. For UDP syslog it is usually 514, for TCP syslog 1468, TCP telnet 21, UDP SNMP 162 (not fully supported yet). If defining your own local port, use a high number between 10,000 and 65,000 to avoid conflicts with other applications. Remote IP Address and IP Port can be left as 0.0.0.0 and 0 for UDP and TCP Server, but must both be specified for TCP Client, as should Retry Attempts (set zero for unlimited retries) and Wait Seconds before next retry attempt. A tick box for each row determines whether the network channel is enabled for capture.

As mentioned earlier, servers can capture from more than one remote client, and should be set-up here by clicking the Add New Row or Copy Row buttons, setting up duplicate protocol and local address and port. The Remote IP Address for on which packets or connections will be filtered may be optionally set-up, if connections are coming from known remote addresses. Currently Remote Filter Information in unused. Network capture channels may be re-ordered using the Up and Down Arrow buttons, and old channels deleted using the Delete Row button. When one or more network channels have been specified, click the main OK button, and the channels will then appear as new tabs in the main ComCap window. Note that further capture settings may need to be specified for specific channels. A maximum of 40 channels may be specified, as a mix of network and serial ports.


2 - Serial Port Capture Functionality

Earlier ComCap releases showed tabs for all installed COM ports and the speed and other settings were done separately in Channel Settings. Now, serial ports are set-up together on the Common Settings Serial Ports tab. Each COM port installed on the PC will appear on this tab, and may be given a Capture Name and enabled or disable for capture. The port Speed, Parity, Data Bits and Stop Bits may be selected from drop down boxes, and Xon/Xoff flow control enabled or disabled.


3 - Main Window Changes

There are now five red/green indicators in the main window that work for both serial and network capture, CTS means the network channel is listening, DSR and DCD that a TCP connection is established, RD that data has just been received and Echo that data is being forwarded to another computer (see later).

The Capture Log title bar shows the configuration for each selected tab, while a panel in the button bar shows the current capture state (or None), and how Echo is configured. The capture channel selection tabs are now coloured, silver when capture is stopped, green if the channel started capture OK, red if capture has not yet started or has stopped. These colours do not mean data is arriving, but that it will be accepted if it arrives. For network channels green means an IP connection is open, for serial that the COM port was opened OK.

The Capture Log right click menu has new options, Word Wrap, Scroll Tabs (rather than multiline if lots of channels are configured), Load Full Log (rather than the partial log normally seen), View Log Window (which shows the whole log in a larger window that does not scroll), Settings, Pause/Resume Capture (stop/start this channel only, tray version only) and Add Comment to Log (tray version only). This last option causes a small dialog will appear, allowing one or more lines to text to be entered, with tick boxes specifying whether blanks lines should be added before or after the text lines. This feature is intended to assist in documenting batch captures, perhaps from laboratory instruments. Note the added text is processed in the same way as any normal captured line, which might mean serial number or time stamps being added.

4 - New General Capture Settings

Capture data line End is now chosen from a menu, Carriage Return, Line Feed (Unix), Special Character specified in hexadecimal, or Packet which is for the UDP protocol where one line is sent per packet without CR or LF. To capture data with both CR and LF, using the CR option causes the LF to be ignored. The Maximum Line Length may be specified up to 1,024 characters (previously it was fixed at 255 characters).

Check for Data Loss must now be enabled for each separate channel to which the common settings should apply.

If capture fails to start, Capture Restart Attempts are specified as seconds before another attempt is made to start capture. This period is also used for restarts used by database problems.

ComCap will now send command strings when capture is started and stopped, perhaps to trigger a remote appliance to start or stop. The strings may include escape sequences to specify non-printing characters:

\n Newline (CRLF)
\f Formfeed (FF)
\c Carriage return (CR)
\l Linefeed (LF)
\\ Backslash (\)
\e Escape (ESC)
\xnn Any hex code where nn is 01 to FF

Note that no line end is normally sent, so \n will commonly be used. A delay in seconds may be specified before the data is send, to allow the connection to settle and perhaps for start-up data to be received.

The data format may now be specified, as Fixed Width Columns, Comma Separated or Variable Named Cols (=), which will be used for saving fields to a database and filtering in a future beta.


5 - New File and Logging Settings

For redundancy, ComCap will now capture to logs on two different disk drives at the same time, and continue logging if one of them is lost. This applies to both the Information Log set-up in Common Settings, and the individual Capture Logs for each channel, with Alternate Directory fields being added (which may be left blank). ComCap detects when disk drives are removed (or fail) or are added, and resets the logs so they pick-up available disks.

A new Logging Type menu specifies whether capture data should be saved into Single Files (as in earlier ComCaps), to a Common File or No Files. The new Common File option allows two or more capture channels to save data into into the same file, with the rotation settings specified in Common Settings. No Files is where data is just being echoed or saved to a database, but is really not recommended.

There is a new capture log rotation interval of 'Multiple Per Day', set as the number of required logs per day from a starting time, so two logs per day from 7am would cause new logs at 7am and 7pm, three logs from 6am would cause logs ay 6am, 2pm and 10pm.

Log Name Format now has 'Named' logs where the log takes the channel display name.

A new Add Escaped Text menu replaces Time Stamp Each Line in previous ComCaps and adds more flexibility. Text may be added at the start or end of the captured line, and may include one or more escape sequences:

\# Serial number, unique to channel incremented for each line
\d Date and time, ie 21-Jun-2006 20:10:12
\t Time, ie 20:10:12
\o ISO date and time, ie 2006-06-21T20:12:11
\\ Backslash
\s Space
\p PC Name (NETBIOS)
\i Local IP address, ie 192.168.44.55
\r Remote IP address
\m Channel Name

The next Serial Number may be specified, as may the fixed number of digits so it has leading zeros, or it may be variable length without leading zeros. If fixed length, the next serial number will wrap to 1 when the length is exceeded, ie 3 digts means 999 to wrap to 1.

More intense validation is now performed on the file directories entered, with a warning if the drive is removable or a network drive. The ComCap Service could previously not use mapped network drives, since these are set-up for interactive logon only. So Common Settings adds a Local Network Logon and Drive Mapping function, allowing a local drive (ie S:) to be mapped to a remote PC name and drive share, with a specific logon and password. This drive is mapped as soon as ComCap starts, and unmapped when it closes. It must not duplicate an existing mapped drive letter. If the ComCap service is used, only this mapped drive will be permitted.

In Common Settings, Log Files, the Information Log may now be rotated Daily, Weekly (Monday) or Monthly (the old default). Disk Space Alerts have been added, so that a warning alert is triggered if spare disk space drops below a specified number of megabytes, with capture then being stopped if it drops below another specified number. Note that disk space checking is done for the common Main and Alternate Directory volumes, and it's assumed that all channels will be saving to the same volumes. The number of repeat attempts to open a file with conflict errors is now specified in Common Settings, Capture Logging, 'Maximum attempts to open log' defaulting to 10 attempts, and these attempts are now about one second apart. Any data captured while the log is still closed is now cached, and written as soon as the log is successfully opened. After failures of double the specified (that is new attempts with a new name), capture will stop. Any errors opening the information log are now logged to a comcap-date.log file in the program directory, as is anything that can not be logged due to the log not being open. The same attempts cycle as for capture log is used to try and ensure the information is opened after errors or conflicts.

6 - New Echo Captured Data Features

Previously, ComCap could only echo captured data to a Windows printer, and many users were frustrated by Windows printer drivers that prevented data being printed while still being captured.

ComCap can now echo captured data directly to serial communications ports or parallel ports to drive printers directly, bypassing the Windows drivers and giving immediate print on impact printers, or full pages when 66 to 72 lines have arrived at a page printer (like a laser). The new Printing tab allows the serial port to chosen from a list (which excludes any configured for capture) and the various serial parameters such as speed to be specified. To avoid overflowing buffers, a 'Only Send if DSR' tick box disables serial port output unless the DSR signal is high. Parallel printing is always to LPT1, but without any control, if the printer is not connected there is no warning. But testing shows that windows does  cache data if the printer becomes temporarily unavailable. For both serial and parallel printers, start-up and stop codes may be sent, using the same escape sequences to specify non-printing characters detailed above. For example, the start and stop code for a Hewlett-Packard LaserJet 4100 are:

Start Codes: \e%-12345X\eE\e&a5L\e&l66F\e(s15H Stop Codes: \eE\e%-12345X

which set a left margin of 5, 66 lines per page, and 15 pitch Courier. The printer must be able to accept data with CRLF as line endings.

ComCap can also echo captured data to the network, using UDP (syslog), TCP Server or TCP Client protocols, similarly to the new network capture detailed earlier. Effectively, this allows ComCap to convert serial data to network protocols, to be captured on remote PCs for viewing or redundancy. Selecting any of the three network echo options displays a new Network tab allowing the local and remote IP address and IP ports, and retry attempts, to be specified, as necessary for the protocol. ComCap will also optionally add syslog headers to echoed lines, similar in format to:

<14>Mar 25 17:03:04 PC09 ComCap:

where <14> is a Priority value where the first 7 bits of the number are a facility code and the last 3 bits are severity, a future release will provide better support for these values. This is followed by the date and time, the computer name and program name. A tick box specifies whether echoed UDP lines should have CRLF added (TCP always has CRLF).


7 - ComCap Start-Up and Service Improvements

Previously the ComCap Service that runs in background (Windows NT4 and later), was a separate application. It was set-up from the Control Panel Services window, and it was not easy to check it was working properly except by viewing the various info and capture logs using a text editor.

This has now been improved so the service version is fully integrated with the interactive system tray application. In Common Settings, the ComCap Operating Mode box has six tick boxes: 'Auto Start Capture' means the tray or service application will start capture as soon as they startup; 'Auto Run Tray Application' will cause the tray application to start as soon as the user logs-on; 'Capture Using Background Service' means ComCap will only capture using the background service, not the tray application, so that clicking the main Start button will run the background service to start capture; 'Allow Monitor Background Service' means that the background service and tray application are linked together, so that the log windows and indicators are updated with what the service is doing, in the same way as when capturing using the tray application; 'Send Alert for Unexpected Stop' will trigger an alert if ComCap stops abnormally. 'Display Lines Captured by Background Service' also needs to be ticked for the background service to echo line to the tray application. When enabling the ComCap Background Service, an optional user account and password may be specified, rather than the service running in the local user account. Running in the local system account has some limitations. Network shared drives might not accessed (so the capture and log files must be on the local PC), no printing is available nor are regional settings available so some dates might be formatted in American style. This option is only available when initially configuring ComCap to use the Background Service, to change the account the service needs to be disabled, ComCap restarted and the service enabled again.

The ComCap Service is now installed or uninstalled after changing the Operating Mode in Common Settings, it is now called 'ComCap Service' rather than 'Magenta ComCap Service' for ComCap Release 3. It should not be necessary to manually change the service properties. There are subtle caption differences when the Capture Using Background Service is configured, the hints on the main Start and Stop buttons will say Start or Stop Capture Service, and the status bar says Configured to Capture Using Background Service. The ComCap Service will normally be configured to auto start, so it starts when the PC is booted, before any users log on. When the user does log-on, the tray application may also auto start (or be started manually) and will automatically detect if the service is running and start monitoring it. Exiting the tray application will leave the service running, unless it's deliberately stopped.

Added the ability to set the Windows priority used by ComCap, this is in Common Settings, Capture Logging, Application Priority, and defaults to 'High'. This means ComCap will run with a higher priority than most other 'Normal' applications, making loss of data due to other application using too much CPU less likely. It is also possible to set ComCap to run as 'Real Time' which means it has higher priority than most Windows functions such as mouse movement, flushing files, etc, but may mean the PC locks up completely if ComCap were to severely misbehave (which is, we hope, unlikely). ComCap has always warned if it detects a temporary PC freeze of five seconds or more, usually caused by another application making heavy use of CPU possibly preventing ComCap capturing and logging data.

ComCap 4 may now be accessed by users without administrator access to the PC, provided file permissions are set-up accordingly for the directories being used by ComCap. Note that only administrators may start and stop Windows services. If ComCap is configured to run as a service, non-administrators will be able to run the system tray application and monitor the service, but not start or stop it (the buttons are hidden). The PC name, user name and security level are logged on start-up, for the service it will say: User: SYSTEM (Administrator).

All ComCap settings are now stored in files, rather than in the registry, to ease sharing between users and back-up. Actual configuration settings are stored in the Windows common application path, which will usually be:

C:\Documents and Settings\All Users\Application Data\Magenta-Systems\ComCap4

in the two files comcap.config and comcap.current. The former file is only updated when configuration changes occur, the latter file contains dynamic information updated ever few seconds, such as log names and serial numbers.

User specific information such as windows sizes and positions, fonts, etc, for the tray application is stored separately for each user:

C:\Documents and Settings\User\Application Data\Magenta-Systems\ComCap4

in a single file comcap.forms. If access to directories is blocked, or they don't exist (Windows 98), then the configuration files are stored in the ComCap program directory.


8 - Windows Vista

ComCap 4 has been successfully tested with Windows Vista RC2, an early version of the next major release of Windows due in early 2007. Vista tightens Windows security considerably, one major change being that programs run by administrators no longer automatically take administrator rights, which may restrict some functionality, such as configuring ComCap to run as a service (the tick boxes will be disabled). When running a program from Explorer in Vista, the right click menu has a 'Run as Administrator' option that should be used when configuring ComCap.


9 - Alerts by Email and SMS

ComCap 4 extends the use of alerts from earlier versions. The Alert Window now supports multiple alerts, which may be cleared individually, and will show alerts when the ComCap service is running, provided the tray application is also running. Alerts may also be sent by email or SMS to mobile phones, this is configured in Common Settings, Email/SMS.

For email, one or two SMTP server host names or IP addresses need to be specified, with the email Subject (ie ComCap Alert) and From Address (ie "ComCap Alert" <comcap@magsys.co.uk>), and one or more email To Addresses (ie "Angus Robertson" <angus@magsys.co.uk>). If the SMTP server needs authentication, this may be selected from Plain, Login or CRAM-MD5 and a User name and Password added. A button allow a test email to be sent, to check the settings.

For SMS to Mobile Telephones, ComCap currently supports sending messages by a internet gateway at Kapow!, http://www.kapow.co.uk/ and using a GSM modem cabled to a PC serial port. The Kapow! gateway requires an account  to be set-up and message credits to be purchased, starting at £8.50 for 100 messages, reducing for larger purchases. By default, messages will be sent with a from address of www.kapow.co.uk, but a personalised address may be purchased if preferred. ComCap takes only a couple of seconds to send an SMS via Kapow, and it will usually be delivered within five seconds (for the UK at least). Once messages have been delivered to Kapow using the HTTP protocol, ComCap checks their status regularly and reports when Kapow has actually sent them successfully. Kapow is configured by specifying one or more mobile telephone numbers in full international format, always including the country code, ie +447891234567, and the account User name and Password. ComCap has been tested with two dedicated GSM Modems, a Nokia 30 and Siemens MC35 (the TC35 is similar), but should work with other GSM modems or mobile phones (with a serial COM port lead) that support the ETSI 07.05 specification. The GSM Modem should be selected from the drop down list of installed modems, which is the only configuration needed. A button allows a test SMS message to be sent. With the GSM Modem, any unread messages on the phone SIM will be downloaded and displayed in the Alert window before the new message is sent, but there is no regular check for new messages. Our testing experience is that the GSM Modems are less reliable than using the Kapow gateway, but do have the benefit of running when there is no internet connection.

In this beta, the following Alerts are supported: 'Disk Space Running Low', 'Disk Space Below Minimum Required', 'PC Temporary Freeze Detected', and 'No Data Received'. Further alerts will be added in the next beta. Alerts may be tested from the File Menu, 'Test Alert', which will send the alert as configured in Common Settings, Alert/Data Loss.


10 - Database and Data Formats

ComCap allows captured data to be saved in a database table. Currently only databases supported by Microsoft Data Access Components (MDAC or ADO) may be used, with testing using Microsoft SQL Servers 2000 and 2005. In theory Access (Jet) could be used, but since SQL Server 2005 Express is free (database limited to 2 gigs), there seems little point. MDAC 2.8 or later (free download from Microsoft) should be installed before attempting to configure database support in ComCap, although this comes standard with Windows XP SP2 and later.  ComCap has also been successfully tested with MySQL 5 using the MySQL ODBC 3.51 driver.

ComCap includes two SQL files, newdb.sql and storedproc.sql that contains SQL 2000 and 2005 statements to create a ComCap SQL database with four new example tables and stored procedures to add records to those tables, and provide limited reporting. newdb.sql will need to be edited to specify the disk volumes for the data and log files. The files will normally be opened in SQL Query Analyser (for SQL 2000) or SQL Server Management Studio (for SQL 2005) and run to create the sample tables and stored procedures.

There are four sample tables in the COMCAP database:

capture_whole - event time and serial, and the whole record in on column
capture_fixed - telephone call data record demo, various call fields
capture_csv - comma separated fields, from ComGen test data
capture_sonicwall - Sonicwall firewall syslog variable named columns

While ComCap can be used to write directly to these, or any other, SQL tables, it is generally recommended that stored procedures are used instead since these allow data validation and manipulation to be performed using SQL functions and commands. For instance, dates and times may be manipulated using SQL string functions into unambiguous formats that SQL will accept, ideally ISO format (yyyy-mm-ddThh:mm:ss.zzz). All stored procedures called by ComCap must return a single row resultset with two columns, retcode and retmess, with retcode set to 100 for success, anything else results in ComCap reporting an error with the retmess.

ComCap handles two columns specially, serial_nr and event_time which should be the first and second columns in the table or parameters in the stored procedure, if used. These extra columns are updated with a serial number and the time the event was added to the database, if specified and are usually the unique key for the table. The serial number is the same as used for adding escaped text to the captured data and the starting number and length may be specified on the Settings Logging tab. While these two special columns will provide a unique database key where there is nothing guaranteed unique in the captured data, it is generally recommended that the serial number and event time are added as escaped text to the captured line, and then selected in the data format as columns for the database, this has the advantage of keeping the logged files the same as the database.

There are six sample stored procedure supplied to put data into the various sample tables: capture_whole_put, capture_fixed_put1 (call_time is HH:MM with no date, call_len as HH:SS), capture_fixed_put2 (call_time is YYMMDD HHMM or MMDDHHMM, call_len is SSS), capture_csv_put, capture_sonicwall_put, capture_sonicwall_put2 (which parses IP addresses and ports into separate columns). There are also xx_lst sample stored procedures to list rows by date range.

To set-up ComCap to save captured data to a database, the SQL table and ideally stored procedure must be created first, to allow ComCap to read the column and parameter definitions. On the Settings, General tab, tick Save to Database and select the Data Format for the data. This may be 'fixed width columns' specified by starting position on line and length,  'character separated columns (CSV)' with a specified separator usually a comma, where columns are counted, or 'variable named columns (=)' where space separated columns are named, ie msg="Connection Opened" n=6258475 src=192.168.1.109:3008:LAN dst=216.22.212.19:80:WAN proto=tcp/http.

On the Settings, Database tab, select database type as ADO and click the Specify Database Button to display the MDAC Data Link Properties dialog. From there, select the OLE DB provider, usually 'for SQL Server', then select or enter a SQL server name, authentication and the database to be used, then click Test Connection to make sure SQL is working, then OK, and the database details will appear on the Database tab. Next select how to add records, either 'insert into table' and select the table from the drop down list, or 'stored procedure' again selected from the drop down list. If the extra columns for serial_nr and/or event_time are being added, this should be selected. 'Pause Capture for Database Problems' will temporarily stop capture if the database becomes unavailable, with attempts to open the database connection and restart capture as specified on the General tab. A database timeout in seconds may be specified, and an inactivity timeout if the database connection should be closed after an idle period, it will be automatically re-opened when more data arrives.

Note that more than one channel can capture data to the same database table, but each channel will use a separate link, to allow saving in parallel. If the database link fails, ComCap will attempt to temporarily save rows that could not be written to the database, and then update the database when it's next opened. Note this data is lost if ComCap is exited or the service version stopped, it's intended for short database interruptions, such as a server or network restart.

Once the database table or stored procedure have been specified, move to the Data Forma tab. If it is empty, or the table or SP have just been changed, click the New Columns button (third from the left) and the grid will be filled with the database column names, types and lengths, or the stored procedure parameters, and will also display sample captured data (if any). This tab dynamically parses captured data according to the data format, so you can easily check how the format you specify will identify data columns for the database. The sample data is taken from anything in the capture window, or clicking the Open Sample Log button allows a specific capture log file to be loaded. The up and down buttons in the second block move up and down the capture log, parsing each line. Note that clicking the New Columns button clears any user data entered. 

The columns that appear in the grid depend on the data format. If this was selected as 'fixed width columns', the SQL column name, type, length and nullable state will be prefilled, and the data position and data length in the captured data need to be specified for each database column to be added. The sample line below the grid shows 150 columns positions. If a database column or parameter is to be left blank, set the data position to 0. Data lengths longer than the column length will be truncated, and all data is trimmed to remove leading and trailing spaces. As the data positions are specified, clicking the Reparse button (right most) will refresh the Sample column, showing the data that would be added to the database column. The same data may be added to more than one column if needed, or in the case of the capture_whole sample table, just a single column is specified for the entire record (the stored procedure could then parse the columns instead). When writing blank data, a NULL is used for blank date/time types, or if the table allows nulls in the column - but note stored procedure parameters are all nullable so numerics are passed as 0 and text as blank.

For data format 'character separated columns (CVS)', the database column name, type, length and nullable are again filled, and only the column sequence needs to be specified. The sample data shows the column sequence number for each parsed data column, allowing the sequence to be easily identified. Specifying the data format 'variable named columns' is similar, except the data column name is specified instead of the sequence. The sample data is parsed to separate the data names from the data values. Note spaces are not allowed in the data names. Any named columns not
found are treated as blank or null.

11 - ComGen - Serial and Network Data Stream Generator

ComGen is a new application delivered free with ComCap 4, designed for testing ComCap. It is designed to generate various types of test streams using any or all of the PC serial COM ports and potentially dozens of network streams, UDP Client, TCP Client or TCP Server. When run, five tabs will be visible, Data Types, Network, Serial, Settings and Progress.

Data Types defines the actual lines of data that will be streamed. Each different type should be given a name, and a format selected, Simple Text, Masked Text, File - Once, File - Repeat. Masked Text may include time and data mask characters formatted in the same way as ComCap capture file names. Both text formats may include one or more escape sequences:

\n Newline (CRLF)
\f Formfeed (FF)
\c Carriage return (CR)
\l Linefeed (LF)
\#6 Serial number, where the digit is the length
\d Date and time, ie 21-Jun-2006 20:10:12
\t Time, ie 20:10:12
\o ISO date and time, ie 2006-06-21T20:12:11
\\ Backslash
\h Syslog header
\p PC Name (NETBIOS)
\i Local IP address, ie 192.168.44.55
\? ComGen channel ID

For the two File formats, the button at the end of the row may be clicked to select a text file to stream. The file may be actual data captured by ComCap such as CDRs (two small sample files are supplied) or any other textual data, optionally including the escape sequences above.

A sample Simple Text format might be 'Text test line from ComGen Id \? on \p at \t serial \#8'.

Network and Serial configuration is very similar to that in ComCap itself, as documented above, specifying UDP Client, TCP Server, TCP Client or the serial COM port parameters. There are additional columns allowing the data frequency, type and serial numbers to be specified. Frequency is selected from the drop down menu as Each Second, Each Minute, Each Hour and Random (between one and 10 seconds) with a multiplier. So 5 by Each Second is five lines of text each second, 30 by Each Minute is one line every two seconds.

The Settings tab allows Auto Start to be specified (this just means the streaming starts as soon as ComGen is run), also network retry attempts allowed (generally leave as zero).

When all the settings are completed, click Start to commence data streaming. The Progress tab will show one row for each stream of data being generated, showing how many lines have been sent and the actual text of the last line. Log window shows the status of the various streams, this is visual only, no disk file is written). ComGen is a low overhead application, and will run for hundreds of hours, generating millions of lines of test data, to thoroughly test ComCap.

1 - Increased the total number of communication ports supported from 20 to 30.  Capturing data on 30 COM ports simultaneously is likely to need a dedicated powerful PC.  ComCap now lists the actual COM port numbers found on start-up in the info log.  ComCap checks for COM1 to COM50, but stops once 30 ports have been found.

 2 - In order to simplify set-up and use, removed the concept of channels being potentially differently numbered from serial port numbers.  This will usually only be noticeable if there are more than two serial ports and they are not numbered sequentially, ie COM3 might be missing but COM4 or higher exists.  ComCap now displays the COM port number on the tabs in the main window and in the logs rather than channel, and it can not be changed.  Previously port 4 could be set to channel 1, and if this was done Capture Settings will need to be changed to match the port.

 3 - Added a new feature to audibly notify newly captured lines of data.  This may be either by the PC beep or by playing a specified sound file.  A gap in seconds may be specified between audible notifications, to prevent continual noise when receiving frequent data.  This feature is really designed for low volume data capture, such as swipe card logging. The new settings for this are on the Capture Settings Logging tab. This feature works with the service version, and may be useful as a gentle confirmation that data is being captured.  A sound file camera.wav is supplied, as an example of what may be played.

 4 - Added a new feature to add a serial number to captured lines of data.  This is primarily for the benefit of other applications that process the data in real time, so they don't process duplicate data.  It may also be used to confirm that lines have not been lost from the capture logs. Note it will not help if data is lost before being written to the logs.  In Settings, the Logging tab has a new option 'Add Serial Number' with options for the start or end of line, the format depending upon the 'Serial Number Mask' and 'Serial Length (digits').  In the mask, the first # symbol is replaced by the serial number, which will be padded with leading zeros if shorter than the specified length, and will wrap back to one when it exceeds the length (max 10 digits or 2 billion).  The 'Next Serial Number' to be used may be specified, and changed if necessary.  For example, a mask of '#|' and length 5 will add '00001|' to the line, while ',"#" would add ',"00001".  If used in conjunction with 'Time Stamp Each Line', the serial will be added before the stamp (if at the start of the line).

 5 - Added a new feature to determine whether trailing spaces should be removed from captured lines of data.  In Settings, the Logging tab has a new option 'Remove Trailing Space' which defaults to ticked.  This is not effective for 'Log Raw Data'.

 6 - Added Capture Settings to the right click menu in the main window, to ease set-up.

 7 - When stopping interactive capture, a confirmation dialog now appear where Yes must be clicked to stop capture.  Previously this dialog only appeared when exiting ComCap with capture still being performed.

 8 - Fixed a possible problem where the interactive window may not have always minimised correctly.

 9 - ComCap now always formats dates in the logs and on the screen with a short alphabetic month, ie Jan, Feb.  This is mainly because the service version always used American date formats in Europe unless the service is set-up to run under a logon account.  This change does not effect the date formatting of any file names or the 'Time Stamp Each Line' option.

 10 - Improved the logging if the capture log file is deleted by another application while ComCap has capture running.  Note that ComCap generally closes the capture file on inactivity so other applications may potentially open it.  The capture log file is always recreated if lost.

 11 - Improved the logging when an error occurs attempting to open the capture log file, making it more obvious there are 10 retries over about 5 seconds.

 12 - Fixed a problem during installation where registry key security rights were only being set-up for four COM ports, not the full 30 supported by ComCap.  This meant the registry security settings previously needed to be edited manually to support more than four COM ports on NT4 and better.

 13 - Now logging errors if the ComCap settings can not be read from the Windows registry, usually due to security rights being insufficient.  This can cause COM ports to not appear, despite being configured.  Re-installing this release of ComCap should allow all ports to be seen (see 12).

 14 - Fixed a problem in Capture Settings where 'Echo to Printer' was sometimes shown disabled when the settings window was displayed, and needed to be ticked again.  The setting was always saved correctly.

 15 - When echoing captured data to a printer, the Common Setting 'Info messages in capture log' is now used to determine whether start and stop information should be printed. 

1 - Increased the maximum number of simultaneous capture channels from eight to 20. Note Magenta Systems has only tested ComCap capturing from three COM ports at once, but one user is capturing from 12 channels simultaneously (using beta 3.0).  Fixed a problem that prevented ComCap accessing two digit serial ports, ie COM10 and higher.  ComCap supports COM1 to COM50.

 2 - Added a New Log interval of Hourly, to create 24 logs per day, on the hour.  This could be used with a custom name of '"capture-"yyyymmdd-hh".txt"' so the file name contains hours only, and no minutes.

 3 - Made a slight change to the New Log interval 'Every x Minutes' so the log is created with zero seconds, ie always on the minute.  This overcomes a time slipping problem that sometimes added one (or more) seconds each time a new log was opened, so the log start time kept changing.

 4 - Added a new logging feature 'Time Stamp Each Line' which adds a time stamp and optional text at the start of each logged line.  This may be useful when logged periodic data such as from alarms that don't include the time within the captured data, or where the time may be inaccurate.  A 'Time Stamp Mask' should be specified, this defaults to 'hh:nn:ss:zzz"| "' which would cause some like '12:41:05:107| ' to precede each captured line.  The ':zzz' can be removed if millisecond accuracy is not needed.  This option is not available if logging 'Raw Data'.  The mask characters and format are identical to the custom file name (look in the help).

 5 - Provided a workaround for a serious potential problem if Windows temporarily freezes due to the misbehaviour of other applications or hardware.  Freezing is normally seen by the mouse being non-responsive and the system tray clock stopping.  The serial PC port hardware only buffers a few characters and relies on Windows drivers reading this data into memory and passing it to ComCap to write to disk.  If one application hogs all the CPU, it may stop Windows and ComCap correctly capturing data, so a warning message is now logged when ComCap has seemingly frozen for more two seconds.  It is recommended the cause of such freezing is determined and the other application stopped.  Sometimes the freezing can be much longer, in particular if there are disk drive problems or if an EIDE tape drive is being used.

 A Hewlett-Packard 20 GB Trevan drive used under Windows XP SP1 causes a test PC to freeze for over two minutes while the tape is being positioned at the start and end of backup, and this is devastating for data capture.  The long freeze not only causes data to be lost, but confuses various internal timers so no more data is captured.  If a freeze of more than 15 seconds (specified in Common Settings) is detected, capture is restarted to overcome this problem.  But data may still have been lost during the freeze.  Fortunately only one ComCap user has reported such data loss, so hopefully freezing is not a widespread problem.

 6 - Added a new feature that allows loss of data to be detected by checking how many minutes since the last data was received.  This will only work where the data flow is reasonably steady, but different warning times may be set for different times of the day and week, for instance with business telephone call logging where there may be no data at night or at the weekends.  Data loss checking is set-up in Common Settings, and is the same for all channels.

 If 'Check for Data Loss' is ticked, a multi-line grid appears allowing several time periods to be specified, each with a different warning period, set in minutes (where zero is no warning).  The default periods are 09:00 Weekdays 5 minutes, 17:00 Weekdays 0 minutes and 00:00 Weekends 0 minutes, which means that the data loss warning is only enabled 9am to 5pm Monday to Friday. Periods may be created for specific days of the week, but if doing so this must be done logically so there are no conflicting periods.  So if Saturday is a working day, periods must be set-up for Saturday and Sunday and there should be no Weekend period.  Clicking in the Valid Days column causes a drop down box of day names to appear. A test button is provided that allows testing of periods for any specific date and time, to ensure there are no conflicts.  If needed, a holiday date list will be added in a later release so warnings are not given unnecessarily.

 If data loss is detected, there are various recovery options. Capture may be restarted in case Windows has itself has got confused, but beware this may cause of data from other active channels since all are restarted.  A warning window may be displayed (interactive version only) and a WAV sound file may be played (a police siren file is defaulted) and repeated after a specified number of seconds (this may be useful for the service version).  Finally a program or batch file may be run optionally with parameters, this could be used for automated email or similar, but beware that interactive programs must not be run by the service version since they will not appear on the desktop and can not be closed.

 7 - Fixed a display problem that meant stopping and restarting capture with more than one channel did not correctly refresh the window for channel 1 with old data.

 8 - Fixed a problem introduced a couple of releases ago that prevented the interactive version of ComCap detecting the service version was already running.  The button bar now clearly indicates the service version is running.

 9 - ComCap may now be accessed by non-administrator users, without needing to change any registry key permissions.

 10 - When using 'Log Name Format' as Standard, time will now always be included in the file name.  Previously a log opened at midnight only had the date.  If a log is only being opened daily (or less often), a custom name mask can be used leaving out the time, ie '"capture-"yyyymmdd".txt"'.  This resolves a potential problem opening file say hourly, when the name format would be different at midnight.

 11 - The 'Channel Settings' and 'Common Settings' property pages have been re-arranged onto multiple tabs so they are less cluttered and easier to find, and to allow functionality expansion.

 12 - Improved the error message when starting capture and finding that the COM port is already being used by another application.

 13 - Improved the error handling when opening a capture log file.  ComCap now makes 10 attempts to open the file, at one second intervals.  After five attempts to open an existing file, the ComCap starts using new file names.  The most likely scenario for an error opening a capture log is if another windows application is being used to process the logs, and it does not use read only shared file access.  Viewing the current log file using Notepad or a proper text editor such as TextPad is always shared access, but using Microsoft Word will lock the file preventing ComCap re-opening it, and returning I/O error 32 or 33.

 14 - The ComCap service version now correctly writes messages to the NT event log.

 15 - The ComCap service version can again be paused and continued, without an error.

 16 – If both ComCap service and interactive versions are run at the same time, the interactive version will indicate in red on the button bar that the service version is running, and prevent capture being started.

2 – Added the ability to create custom capture log file names. These are creating using mask characters that specify how the log date and optional time are formatting in the file name. Be careful creating the file mask, it must include sufficient date and time masks to ensure a unique file name is created to match how often new logs are opened, so a daily log must have the day, an hourly log the time. Any text to appear in the file name should be within single (') or double quotes (").

c Date and time using the Windows default settings.
d Day as a number without a leading zero (1-31).
dd Day as a number with a leading zero (01-31).
ddd Day as an abbreviation (Sun-Sat).
dddd Day as a full name (Sunday-Saturday).
ddddd Date using the short Windows default setting.
dddddd Date using the long Windows default setting.
m Month as a number without a leading zero (1-12).
mm Month as a number with a leading zero (01-12).
mmm Month as an abbreviation (Jan-Dec).
mmmm Month as a full name (January-December).
yy Year as a two-digit number (00-99).
yyyy Year as a four-digit number (0000-9999).
h Hour without a leading zero (0-23).
hh Hour with a leading zero (00-23).
n Minute without a leading zero (0-59).
nn Minute with a leading zero (00-59).
s Second without a leading zero (0-59).
ss Second with a leading zero (00-59).
t Time using the short Windows default setting.
tt Time using the long Windows default setting
am/pm The am/pm specifier can use lower, upper, or mixed case.
a/p The a/p specifier can use lower, upper, or mixed case.
/ Date separator character.
: Time separator character.

For example, the mask "capture-"yyyymmdd"-"hhnnss".txt" (the quotes are very important) will result in the default file name with the format 'capture-20020828-145404.txt'. But the month could be alphabetic, or date and time separation may be added. Clicking the Test button will show how the mask will be interpreted.

3 - Added a new Common Setting, 'Close log after each line is written', which does exactly as it suggests. This is really designed for low speed data capture, since there is a system overhead to opening and closing the capture log file. But it will be useful for those users that have another application needing to access the capture log file. If enabled, the other two options for closing the log are disabled.

4 - Changed the 'Flush capture logs to disk periodically' option so the setting is now in seconds rather than minutes, defaulting to 120 seconds and with a minimum of one second.

5 – Added a start-up check to show a warning dialog if a user without the necessary permissions attempts to run ComCap. When initially installed, only administrator level users can access ComCap (on NT4 and better). To allow normal users access, it is necessary to add extra permissions to the ComCap registry keys, which is explained in the readme under installation and in the help file.

6 - Added a date check to ensure that the last written log date is not later than today, for the rare case where the PC clock gets accidentally set into the future.

7 – Added a new ‘Test RS232 Signals’ utility, which may be useful for simple communications port testing. It allows any installed windows serial communications port to be open, shows the status of the five RS232 signal lines and whether any data is being received, and allows the DTR and RTS control lines to be raised or lowered. If a modem is connected to the serial port and it has front panel indicators, toggling DTR will cause TR to flash, confirming that the modem is cabled correctly to the PC. If a null modem cable is used to connect two PCs each running the utility, the opposing DSR and CTS indicators will appear as DTR and RTS are toggled on each PC. This utility does not show or send any data, it’s just designed to ease resolving cabling problems, or even identifying which PC back panel connector is which communications port.

8 - There is a version jump to 2.6, because the last release was called 2.4 and 2.5 in different places.

9 – ComCap now has a proper help file.

2 - It is now possible to capture log files free of any extra ComCap comments. There is a new information log window beside the main data capture window. A splitter allows the widths of the two windows to be adjusted. All ComCap informational messages are now displayed in this new window and written to a new monthly log file, kept in a directory specified in Common Settings with the file name format 'info-yyyy-mm.txt', ie 'info-2002-03.txt'. The new log also contains the names of all capture log files, with their sizes and number of lines added and file size when closed. Common Settings includes new options 'Info Messages on Capture Log' which effectively allows backward compatibility with earlier releases, and 'Log Diagnostic Messages' which should not be needed unless there are logging problems.

3 - Added a new Common Setting 'Flush capture logs to disk periodically' which specifies that capture logs will be closed every so many minutes flushing them to disk. If the PC subsequently locks up (as an old P133 running Windows 98 did during testing), less information will now be lost. Note this setting is separate to the inactivity delay that closes the log file, it's designed for when data is arriving continually and there is no inactivity.

4 - Added the option to create a new log each time the 'Inactivity Delay' on new data expires. Note a new log is also created after the new 'Flush capture logs to disk periodically' expires so care may be needed choosing the two settings.

5 - Added the ability to optionally create a new capture log file if the captured data contains a form feed or new page control character. Note the new file will have a date/time file name, so there is a restriction of one new page per second. It may be necessary to reduce bits/sec if this becomes an issue so each page takes at least one second.

6 - A lot of users have been frustrated during set-up of ComCap where the data indicator shows something arriving, but nothing is ever displayed. This usually happened because ComCap buffers each line of data before displaying it, waiting for a carriage return control character to signify end of line. If the bits/sec were incorrect, CR was never seen and nothing was displayed. A maximum of 253 characters are now buffered, so random data (due to incorrect bits/sec) should also appear. The CR check may also have caused problems with file created by UNIX systems, which typically only have line feed characters, so a new option 'Line Feed end of Line' has been added for each channel. This might solve a couple of cases where no data has been seen. Normal ASCII files have both CR and LF, so either option will work.

7 - If the directories for info or capture logs are not specified or are not available, ComCap has always placed the logs in the program directory instead. It now also logs the original error in the new info log.

8 - The main capture log window is now only updated while visible. If the ComCap window is minimised or closed, data is still captured to file but not to the non-displayed log window. This avoids needing to clear the log window periodically as it overflows. When the ComCap window is opened, the log window is refresh from disk.

9 - Added an option 'Auto Scroll' to right click menu in the main capture window. This stop ComCap forcing scrolling to the end of the window before adding a new line of captured data, but the behaviour will depend upon the version of Windows. It does not seem to have any effect on W2K, but with Windows 98 the log window will flicker less.

10 - Improved the behaviour when ComCap is run unregistered. A warning in info log window shows that capture will cease after 15 minutes, and when that happens the Start button disappears, rather than ComCap closing down completely, so the captured data can still be seen.

11 - Successfully tested ComCap with a Lava Computer DSerial PCI card which adds an extra two serial ports, COM4 and COM5 on my test PC (COM3 is an USB serial port). Lava also sells four and eight port serial cards, which should also work with ComCap, http://www,lavalink.com/.

12 - This release has been tested over several days with 20 to 30 megs of CDRs being captured daily, on both Windows 98 and 2000. The Windows 2000 PC were doing a dozen other tasks while capturing data, peak memory usage was only 4.5 Mbytes, CPU usage was 1% over 40 hours (23 minutes) on a PIII/850. The capture data source was the Plessey Call Simulator that creates CDRs for three types of large switch (BTS 7#2, iSDX, Ericsson). The Call Simulator outputs data from a serial port, so a null modem cable will be needed to feed the ComCap port.

13 - Apologies for the quality of the documentation for ComCap, it is getting more complicated with each release and this readme is no longer really suitable. A proper help file will be added real soon.

2 - The opening of new logs has always been difficult to test, since it needed ComCap to be running for days or weeks to see if the new log was correctly opened. There were reports of problems in 2.2 opening logs at other than 00:00, and at 00:00 in 2.1. So it was time for a redesign to make this all more reliable. The next log time is now determined when capture starts and shown after the current file name in the window or in the Event Log for the service version, so it's immediately obvious whether ComCap is configured and working correctly. In addition, a new option has been added to open a new log every X minutes (which is also very convenient for testing). New logs are now opened irrespective of data being received.

3 - Now ensure that the log directory has been created when opening logs.

2 - Added a new Common Settings window. This now includes the 'Auto Start Capture' option that was previously on the icon right click menu.

3 - Added a new Common Setting to change the inactivity period before the log file is closed. Closing the file forces captured data to be flushed to disk. Previously the period for fixed at 120 seconds, but may now be set down to one second or up to 999 seconds. The shorter the time, the safer the data, but there's always an overhead opening and closing files.

4 - Added two new Common Settings that reflect how much data the log window shows, and how often it's cleared. 'Maximum log lines to display' defaults to 10,000 lines to restrict the amount of memory used by CompCap, but may be set between 100 and 9,999,99 lines. If the log becomes larger than specified, it is automatically reduced in size by 'Lines to remove when display overflows', which must be less lines than the maximum. The setting used in previous releases has varied, but the 2.1 cleared all lines when 10,000 were reached, which was not very satisfactory.

5 - The log display window right click menu includes a new 'Stop Update' tick option. This is always disabled when capture starts. but if ticked will temporarily stop new captured data being added to the log window, allowing earlier data to be located. Unticking the option will refresh the log display with any new data captured.

6 - Fixed a display problem that caused lines containing null characters to be truncated. Null characters are also now removed from the log file, unless 'Log Raw Data' is ticked.

7 - Fixed a problem introduced in 2.1 that meant a new log was not always created each day if 'New Log Time' was set as 00:00. The same problem meant daily log file names also included the time.

2 - Fixed a major bug introduced in 2.0 that prevented capture with parameters other than 8 data bits and 1 stop bit (which is fortunately what most systems use).

3 - A new option 'New Log Time' has been added that allows logs to be restarted at a time other than midnight. If the time is specified as a time other than 00:00:00, the log file name will also include the time. Only one log per calendar date will be created, so if this option is set before the specified time, it will not create a log file dated 'yesterday'.

Please note that capture has only been tested to two simultaneous serial ports, since more ports are rare, and even that needed a birds nest of null modems, gender benders, 25-9 pin converters and cables to link two PCs together.

2 - 'Auto Capture' has moved from the Settings window to a tick option on the icon menu, since it applies to all ports.

3 - Unless 'No Log Display' is set, ComCap now displays data from the current log file when started. The current log file name is now displayed in Settings. These features make it easier for the interactive ComCap to be used to view the logs from the service version.

4 - The ComCap distribution includes a small application written by a user, 'Concatenation Utility', that allows multiple text file captured by ComCap to be combined into a single text file that may be easier to import into a telephone call management application.

5 - A bug has been fixed in the service version of ComCap, so that the capture file name is now only written to the event log when capture starts, rather than each time it is opened (the log closes automatically after two minutes of inactivity).

6 - An About window has been added.