Adding support for regex in unitTagsFile

9 files changed, 73 insertions, 47 deletions

diff --git a/docs/CONFIGURE.md b/docs/CONFIGURE.md
index 2c777b84..c2b292c6 100644
--- a/docs/CONFIGURE.md
+++ b/docs/CONFIGURE.md
@@ -26,31 +26,31 @@ The amount of tuning error is -14500Hz, so that would go under **error:** for th
 
 **NOTE:** In some instances, an alternative is to use `ppm` correction rather than the `error` configuration option.
 
-Alternatively, you can use this tool here: http://garvas.org/trunk-recorder/ to obtain RTL-SDR dongle/array configurations. 
+Alternatively, you can use this tool here: http://garvas.org/trunk-recorder/ to obtain RTL-SDR dongle/array configurations.
 
 ### Gain
 
-After you have figured out the amount of tuning error, you now need to find the optimal amount of receiver gain to use. 
+After you have figured out the amount of tuning error, you now need to find the optimal amount of receiver gain to use.
 
-Gain is a measure of how much amplification is required for the received signal, and on some SDRs, there are multiple places along the receive path where a signal can be amplified. 
+Gain is a measure of how much amplification is required for the received signal, and on some SDRs, there are multiple places along the receive path where a signal can be amplified.
 
-If there is not enough gain, the signal will not be strong enough to decode, and Trunk-Recorder will fail to lock to the control channel. If there is too much gain, it can distort the signal, there is also the chance you might be causing harm to your SDR reception device. Setting the gain too high will result in amplification of the background RF and create noise. 
+If there is not enough gain, the signal will not be strong enough to decode, and Trunk-Recorder will fail to lock to the control channel. If there is too much gain, it can distort the signal, there is also the chance you might be causing harm to your SDR reception device. Setting the gain too high will result in amplification of the background RF and create noise.
 
-Generally, you can mess around with the gain slider in GQRX until the signal looks well defined and there isn't too much noise. If it is impossible to get a well-defined signal, it could be a sign that you have one or more issues: a better antenna that is tuned to the needed frequency range, moving the antenna to a new location, or using a different SDR device. There could also be some strong interference nearby, which can introduce a lot of background noise making it tough to distinguish the signal. Various computer hardware, poorly grounded hardware, and cheaply made USB hubs can be notorious for producing RF noise across the entire spectrum. 
+Generally, you can mess around with the gain slider in GQRX until the signal looks well defined and there isn't too much noise. If it is impossible to get a well-defined signal, it could be a sign that you have one or more issues: a better antenna that is tuned to the needed frequency range, moving the antenna to a new location, or using a different SDR device. There could also be some strong interference nearby, which can introduce a lot of background noise making it tough to distinguish the signal. Various computer hardware, poorly grounded hardware, and cheaply made USB hubs can be notorious for producing RF noise across the entire spectrum.
 
 Once you find the correct gain settings, use them for this source in the `config.json` file.
 
 ### Center Frequency
-When you set the center frequency for a source, **you are picking the frequency that will be in the _middle_ of the block of spectrum that you are recording**. Half of the bandwidth for the device will be above that frequency and the other half below. 
+When you set the center frequency for a source, **you are picking the frequency that will be in the _middle_ of the block of spectrum that you are recording**. Half of the bandwidth for the device will be above that frequency and the other half below.
 
-For example, if you are using a HackRF, with 8MHz of bandwidth, and you tune the center frequency to 854MHz, it would cover from 850.0MHz to 858.0MHz. 
+For example, if you are using a HackRF, with 8MHz of bandwidth, and you tune the center frequency to 854MHz, it would cover from 850.0MHz to 858.0MHz.
 
 To find your ideal center frequency, look at what the lowest frequency you want to cover is and what the highest is. You need to need to be able cover slightly beyond the frequncy of a channel. This is because the frequency is for the center of the channel and the actual channel is wider and a bit of filtering is done to receive it. The sample rate should be higher than the difference between the low and high frequency. Most SDRs do not perform as well right at the beginnging and end of the frequency range they are set to. It is best to set a slightly higher sample rate than needed, to avoid those spots. Also, some SDRs have some artifacts right at there center frequency, so ensure that center frequency doesn't land on the frequency of a channel you are trying to record.
 
 ### Multiple Sources
-If the low frequency and high frequency of the system you are trying to capture is greater than the amount of bandwidth your SDR can capture, you need to use multiple SDRs. 
+If the low frequency and high frequency of the system you are trying to capture is greater than the amount of bandwidth your SDR can capture, you need to use multiple SDRs.
 
-In addition to being able to use a cheaper SDR, it also helps with general performance of the devices. When a single SDR is used, each of the Recorders gets fed all of the sampled signals. Each Recorder needs to cut down the multi-mega samples per second into a small 12.5Khz or even 6.25Khz(!) slivers. 
+In addition to being able to use a cheaper SDR, it also helps with general performance of the devices. When a single SDR is used, each of the Recorders gets fed all of the sampled signals. Each Recorder needs to cut down the multi-mega samples per second into a small 12.5Khz or even 6.25Khz(!) slivers.
 
 When you use multiple SDRs, each SDR is capturing only a partial slice of the system so the recorders have to cut down a much smaller amount of sample to get to the sliver they are interested in. This ultimately denotes that you can have a lot more recorders running!
 
@@ -91,11 +91,11 @@ Here is a map of the different sections of the *config.json* file:
 
 ```json
 {
-	Global Configs
-	
-	"sources": [{ Source Object }, { Source Object }],
-	"systems": [{ System Object }, { System Object }],
-	"plugins": [{ Plugin Object }]
+  Global Configs
+
+  "sources": [{ Source Object }, { Source Object }],
+  "systems": [{ System Object }, { System Object }],
+  "plugins": [{ Plugin Object }]
 }
 ```
 
@@ -179,9 +179,9 @@ Here is a map of the different sections of the *config.json* file:
 | analogLevels           |          | 8                              | number (1-32)                                                | The amount of amplification that will be applied to the analog audio. |
 | maxDev                 |          | 4000                           | number                                                       | Allows you to set the maximum deviation for analog channels. If you analog recordings sound good or if you have a completely digital system, then there is no need to touch this. |
 | digitalLevels          |          | 1                              | number (1-16)                                                | The amount of amplification that will be applied to the digital audio. |
-| unitTagsFile           |          |                                | string                                                       | This is the filename of a CSV files that provides information about the unit tags. It allows a Unit ID to be assigned a name. This file should be located in the same directory as the trunk-recorder executable. The format is 2 columns, the first being the decimal number of the Unit ID, the second is the Unit Name, |
+| unitTagsFile           |          |                                | string                                                       | This is the filename of a CSV file that provides information about the unit tags. The format for the file is described below. |
 | recordUnknown          |          | true                           | **true** / **false**                                         | Record talkgroups if they are not listed in the Talkgroups File. |
-| recordUUVCalls         |          | true                           | **true** / **false**                                         | [ P25 only ] Record Unit to Unit Voice calls. 		     |
+| recordUUVCalls         |          | true                           | **true** / **false**                                         | [ P25 only ] Record Unit to Unit Voice calls.        |
 | hideEncrypted          |          | false                          | **true** / **false**                                         | Hide encrypted talkgroups log entries                        |
 | hideUnknownTalkgroups  |          | false                          | **true** / **false**                                         | Hide unknown talkgroups log entries                          |
 | minDuration            |          | 0<br />(which is disabled)     | number                                                       | The minimum call duration in seconds (decimals allowed), calls below this number will have recordings deleted and will not be uploaded. |
@@ -251,9 +251,9 @@ This plugin makes it easy to connect Trunk Recorder with [Rdio Scanner](https://
 **Name:** simplestream
 **Library:** libsimplestream.so
 
-This plugin streams uncompressed audio (16 bit Int, 8 kHz, mono) to UDP or TCP ports in real time as it is being recorded by trunk-recorder.  It can be configured to stream audio from all talkgroups and systems being recorded or only specified talkgroups and systems.  TGID information can be prepended to the audio data to allow the receiving program to take action based on the TGID.  Audio from different Systems should be streamed to different UDP/TCP ports to prevent crosstalk and interleaved audio from talkgroups with the same TGID on different systems.  
+This plugin streams uncompressed audio (16 bit Int, 8 kHz, mono) to UDP or TCP ports in real time as it is being recorded by trunk-recorder.  It can be configured to stream audio from all talkgroups and systems being recorded or only specified talkgroups and systems.  TGID information can be prepended to the audio data to allow the receiving program to take action based on the TGID.  Audio from different Systems should be streamed to different UDP/TCP ports to prevent crosstalk and interleaved audio from talkgroups with the same TGID on different systems.
 
-This plugin does not, by itself, stream audio to any online services.  Because it sends uncompressed PCM audio, it is not bandwidth efficient and is intended mostly to send audio to other programs running on the same computer as trunk-recorder or to other computers on the LAN.  The programs receiving PCM audio from this plugin may play it on speakers, compress it and stream it to an online service, etc.  
+This plugin does not, by itself, stream audio to any online services.  Because it sends uncompressed PCM audio, it is not bandwidth efficient and is intended mostly to send audio to other programs running on the same computer as trunk-recorder or to other computers on the LAN.  The programs receiving PCM audio from this plugin may play it on speakers, compress it and stream it to an online service, etc.
 
 **NOTE 1: In order for this plugin to work, the audioStreaming option in the Global Configs section (see above) must be set to true.**
 
@@ -271,11 +271,11 @@ This plugin does not, by itself, stream audio to any online services.  Because i
 | port      |    ✓     |               | number | UDP or TCP port that this stream will send audio to. |
 | TGID      |    ✓     |               | number | Audio from this Talkgroup ID will be sent on this stream.  Set to 0 to stream all recorded talkgroups. |
 | sendTGID  |           |    false     | boolean | When set to true, the TGID will be prepended in long integer format (4 bytes, little endian) to the audio data each time a packet is sent. |
-| shortName |          |              |string  | shortName of the System that audio should be streamed for.  This should match the shortName of a system that is defined in the main section of the config file.  When omitted, all Systems will be streamed to the address and port configured.  If TGIDs from Systems overlap, each system must be sent to a different port to prevent interleaved audio for talkgroups from different Systems with the same TGID.  
+| shortName |          |              |string  | shortName of the System that audio should be streamed for.  This should match the shortName of a system that is defined in the main section of the config file.  When omitted, all Systems will be streamed to the address and port configured.  If TGIDs from Systems overlap, each system must be sent to a different port to prevent interleaved audio for talkgroups from different Systems with the same TGID.
 |  useTCP   |        |   false     |boolean | When set to true, TCP will be used instead of UDP.
 
 ###### Plugin Object Example #1:
-This example will stream audio from talkgroup 58914 on system "CountyTrunked" to the local machine on UDP port 9123.  
+This example will stream audio from talkgroup 58914 on system "CountyTrunked" to the local machine on UDP port 9123.
 ```yaml
         {
           "name":"simplestream",
@@ -347,11 +347,11 @@ pulseaudio is the default sound system on many Linux computers, including the Ra
 
 An example command to set up pulseaudio to receive 8 kHz audio (digital audio) from simplestream on TCP port 9125:
 ```
-pacmd load-module module-simple-protocol-tcp sink=1 playback=true port=9125 format=s16le rate=8000 channels=1 
+pacmd load-module module-simple-protocol-tcp sink=1 playback=true port=9125 format=s16le rate=8000 channels=1
 ```
 An example command to set up pulseaudio to receive 16 kHz audio (analog audio) from simplestream on TCP port 9125:
 ```
-pacmd load-module module-simple-protocol-tcp sink=1 playback=true port=9125 format=s16le rate=16000 channels=1 
+pacmd load-module module-simple-protocol-tcp sink=1 playback=true port=9125 format=s16le rate=16000 channels=1
 ```
 The matching simplestream config to send audio from talkgroup 58918 to TCP port 9125 would then be something like this:
 ```yaml
@@ -382,16 +382,16 @@ The Trunk Record program really only uses the priority information and the Dec T
 
 Here are the column headers and some sample data: NOTE: If you are adding the Priority to a RR csv, as well as changing order you must use a heading for the first column other than "Decimal" eg DEC for TR to detect you are supplying this layout.
 
-| DEC |	HEX |	Mode |	Alpha Tag	| Description	| Tag |	Group | Priority |
-|-----|-----|------|-----------|-------------|-----|-------|----------|
-|101	| 065	| D	| DCFD 01 Disp	| 01 Dispatch |	Fire Dispatch |	Fire | 1 |
-|2227 |	8b3	| D	| DC StcarYard	| Streetcar Yard |	Transportation |	Services | 3 |
+| DEC  | HEX | Mode | Alpha Tag    | Description    | Tag            | Group    | Priority |
+|------|-----|------|--------------|----------------|----------------|----------|----------|
+| 101  | 065 | D    | DCFD 01 Disp | 01 Dispatch    | Fire Dispatch  | Fire     | 1        |
+| 2227 | 8b3 | D    | DC StcarYard | Streetcar Yard | Transportation | Services | 3        |
 
 
 
 ## channelFile
 
-This file allows for you to specify additional information about conventional channels. A recorder is started for each line in the file and set the to frequency specified. The type of recorder is based on the type of System. A *Conventional* system would have Analog Recorders, while a *ConventionalP25* or *ConventionalDMR* would have digital recorders. 
+This file allows for you to specify additional information about conventional channels. A recorder is started for each line in the file and set the to frequency specified. The type of recorder is based on the type of System. A *Conventional* system would have Analog Recorders, while a *ConventionalP25* or *ConventionalDMR* would have digital recorders.
 
 *Tone based squelch is currently not supported.*
 
@@ -401,3 +401,18 @@ The **Enable** Column is optional and defaults to *True*. It only needs to be ad
 | --------- | --------- | -------- | ------------- | ---------------------- | ------ | ------ | ------------------- |
 | 300       | 462275000 | 94.8 PL  | Town A Police | Town A Police Dispatch | Police | Town A |                     |
 | 325       | 462275000 | 151.4 PL | Town B DPW    | Town B Trash Dispatch  | DPW    | Town B | False               |
+
+
+## unitTagsFile
+
+This file allows for Unit IDs to be assigned a name. The format is 2 columns, the first being the decimal number of the Unit ID, the second is the Unit Name.
+
+Regex is also supported for the Unit ID, which can be used to match radio IDs of a specific pattern. By default, the regex must match the full string (`^pattern$`), however putting the pattern within `/` will allow partial matches. Within the unit name, `$1`, `$2`, etc. will be replaced by the corresponding capture group. For large radio systems, regex may be better instead of specifying a long list of radio IDs. In case a Unit ID will be matched by regex but you do not want to use the associated unit name, you can put the specific unit ID and unit name before the regex, so it will be chosen before reaching the regex.
+
+In the second row of the example below, the first capture group `([0-9]{2})` becomes `$1` for the unit name, so an ID like 1210207 gets translated to Engine 20. In the third row, only the start of the string is being matched, so an ID of 173102555 is translated into Ambulance 102.
+
+| Unit ID                  | Unit Name    |
+| ---------                | ---------    |
+| 911000                   | Dispatch     |
+| 1[1245]10([0-9]{2})[127] | Engine $1    |
+| /^1[78]3(1[0-9]{2})/     | Ambulance $1 |
diff --git a/trunk-recorder/call_concluder/call_concluder.cc b/trunk-recorder/call_concluder/call_concluder.cc
index 00d28f3d..ff891c2d 100644
--- a/trunk-recorder/call_concluder/call_concluder.cc
+++ b/trunk-recorder/call_concluder/call_concluder.cc
@@ -287,8 +287,7 @@ Call_Data_t Call_Concluder::create_call_data(Call *call, System *sys, Config con
       call_info.stop_time = t.stop_time;
     }
 
-    UnitTag *unit_tag = sys->find_unit_tag(t.source);
-    std::string tag = (unit_tag == NULL || unit_tag->tag.empty() ? "" : unit_tag->tag);
+    std::string tag = sys->find_unit_tag(t.source);
     Call_Source call_source = {t.source, t.start_time, total_length, false, "", tag};
     Call_Error call_error = {t.start_time, total_length, t.length, t.error_count, t.spike_count};
     call_info.transmission_source_list.push_back(call_source);
diff --git a/trunk-recorder/systems/system.h b/trunk-recorder/systems/system.h
index 098c73c5..5f4b4b66 100644
--- a/trunk-recorder/systems/system.h
+++ b/trunk-recorder/systems/system.h
@@ -108,7 +108,7 @@ public:
   virtual void set_source(Source *) = 0;
   virtual Talkgroup *find_talkgroup(long tg) = 0;
   virtual Talkgroup *find_talkgroup_by_freq(double freq) = 0;
-  virtual UnitTag *find_unit_tag(long unitID) = 0;
+  virtual std::string find_unit_tag(long unitID) = 0;
   virtual void set_talkgroups_file(std::string) = 0;
   virtual void set_channel_file(std::string channel_file) = 0;
   virtual bool has_channel_file() = 0;
diff --git a/trunk-recorder/systems/system_impl.cc b/trunk-recorder/systems/system_impl.cc
index f0aabc37..02606ed5 100644
--- a/trunk-recorder/systems/system_impl.cc
+++ b/trunk-recorder/systems/system_impl.cc
@@ -306,7 +306,7 @@ Talkgroup *System_impl::find_talkgroup(long tg_number) {
 Talkgroup *System_impl::find_talkgroup_by_freq(double freq) {
   return talkgroups->find_talkgroup_by_freq(sys_num, freq);
 }
-UnitTag *System_impl::find_unit_tag(long unitID) {
+std::string System_impl::find_unit_tag(long unitID) {
   return unit_tags->find_unit_tag(unitID);
 }
 
diff --git a/trunk-recorder/systems/system_impl.h b/trunk-recorder/systems/system_impl.h
index 63850f36..24c042cb 100644
--- a/trunk-recorder/systems/system_impl.h
+++ b/trunk-recorder/systems/system_impl.h
@@ -166,7 +166,7 @@ public:
   void set_source(Source *);
   Talkgroup *find_talkgroup(long tg);
   Talkgroup *find_talkgroup_by_freq(double freq);
-  UnitTag *find_unit_tag(long unitID);
+  std::string find_unit_tag(long unitID);
   void set_talkgroups_file(std::string);
   void set_channel_file(std::string channel_file);
   bool has_channel_file();
diff --git a/trunk-recorder/unit_tag.cc b/trunk-recorder/unit_tag.cc
index d39908b7..182a3781 100644
--- a/trunk-recorder/unit_tag.cc
+++ b/trunk-recorder/unit_tag.cc
@@ -1,6 +1,7 @@
 #include "unit_tag.h"
+#include <boost/regex.hpp>
 
-UnitTag::UnitTag(long num, std::string t) {
-  number = num;
+UnitTag::UnitTag(std::string p, std::string t) {
+  pattern = p;
   tag = t;
 }
diff --git a/trunk-recorder/unit_tag.h b/trunk-recorder/unit_tag.h
index d888f805..6850dd79 100644
--- a/trunk-recorder/unit_tag.h
+++ b/trunk-recorder/unit_tag.h
@@ -4,14 +4,14 @@
 #include <iostream>
 #include <stdio.h>
 #include <string>
-//#include <sstream>
+#include <boost/regex.hpp>
 
 class UnitTag {
 public:
-  long number;
+  boost::regex pattern;
   std::string tag;
 
-  UnitTag(long num, std::string t);
+  UnitTag(std::string p, std::string t);
 };
 
 #endif // UNIT_TAG_H
diff --git a/trunk-recorder/unit_tags.cc b/trunk-recorder/unit_tags.cc
index e8455d8c..b3ebda70 100644
--- a/trunk-recorder/unit_tags.cc
+++ b/trunk-recorder/unit_tags.cc
@@ -5,6 +5,7 @@
 #include <boost/intrusive_ptr.hpp>
 #include <boost/log/trivial.hpp>
 #include <boost/tokenizer.hpp>
+#include <boost/regex.hpp>
 
 #include "csv_helper.h"
 #include <cstdio>
@@ -60,7 +61,7 @@ void UnitTags::load_unit_tags(std::string filename) {
       continue;
     }
 
-    add(atoi(vec[0].c_str()), vec[1].c_str());
+    add(vec[0].c_str(), vec[1].c_str());
 
     lines_pushed++;
   }
@@ -75,21 +76,31 @@ void UnitTags::load_unit_tags(std::string filename) {
   }
 }
 
-UnitTag *UnitTags::find_unit_tag(long tg_number) {
-  UnitTag *tg_match = NULL;
+std::string UnitTags::find_unit_tag(long tg_number) {
+  std::string tg_num_str = std::to_string(tg_number);
+  std::string tag = "";
 
   for (std::vector<UnitTag *>::iterator it = unit_tags.begin(); it != unit_tags.end(); ++it) {
     UnitTag *tg = (UnitTag *)*it;
 
-    if (tg->number == tg_number) {
-      tg_match = tg;
+    if (regex_match(tg_num_str, tg->pattern)) {
+      tag = regex_replace(tg_num_str, tg->pattern, tg->tag, boost::regex_constants::format_no_copy);
       break;
     }
   }
-  return tg_match;
+
+  return tag;
 }
 
-void UnitTags::add(long num, std::string tag) {
-  UnitTag *unit_tag = new UnitTag(num, tag);
+void UnitTags::add(std::string pattern, std::string tag) {
+  // If the pattern is like /someregex/
+  if (pattern.substr(0, 1).compare("/") == 0 && pattern.substr(pattern.length()-1, 1).compare("/") == 0) {
+    // then remove the / at the beginning and end
+    pattern = pattern.substr(1, pattern.length()-2);
+  } else {
+    // otherwise add ^ and $ to the pattern e.g. ^123$ to make a regex for simple IDs
+    pattern = "^" + pattern + "$";
+  }
+  UnitTag *unit_tag = new UnitTag(pattern, tag);
   unit_tags.push_back(unit_tag);
 }
diff --git a/trunk-recorder/unit_tags.h b/trunk-recorder/unit_tags.h
index a048214b..5f95df28 100644
--- a/trunk-recorder/unit_tags.h
+++ b/trunk-recorder/unit_tags.h
@@ -12,7 +12,7 @@ class UnitTags {
 public:
   UnitTags();
   void load_unit_tags(std::string filename);
-  UnitTag *find_unit_tag(long unitID);
-  void add(long num, std::string tag);
+  std::string find_unit_tag(long unitID);
+  void add(std::string pattern, std::string tag);
 };
 #endif // UNIT_TAGS_H