Before you begin your database construction, it’s important to understand the types of data you will receive, and how that information can be categorized into the data format of Concordance Desktop fields.
Concordance Data Field Types
There are two basic field categories available in the Concordance Desktop system: defined length fixed fields and variable length paragraphs. The full-text paragraph fields are the most flexible and most commonly used type in a Concordance Desktop database.
Fixed fields are always a pre-defined length and contain numbers, dates, or short amounts of text. Documents in a Concordance Desktop database can be sorted by the contents of both fixed fields and free-text paragraph fields, but fixed length fields sort much faster. Data in fixed fields is typically searched using relational comparison search operators such as =, >, and <, however fixed fields can be fully indexed and searched using full-text techniques as well.
Fixed length fields have unique applications when compared to full-text paragraphs. While paragraph fields are flexible and variable in size, they do not sort quickly or search easily by comparison.
•Fixed-length fields sort the fastest.
•Fixed fields display faster in Table view than Paragraphs. This is useful in long document databases.
•Relational searches for dates and numbers work best when they are stored in Date and Numeric fields.
•Numeric and date math using the Report Writer or using Concordance’s programming language work best with Date and Numeric fields.
Use the following table as a reference when you review your data
Data Type |
Capacity |
Type |
Notes |
|---|---|---|---|
Date |
8 bytes |
Fixed |
Just for dates, keyed by default |
Numeric |
1-20 digits |
Variable |
Currency, zero-filled, comma, keyed by default |
Text |
1-60 characters |
Variable |
Alphanumeric, keyed by default |
Paragraph |
12 million characters |
Fixed |
Alphanumeric, indexed by default, allows rich-text format |
|
If you want a field to be full-text searchable, make it a 'Paragraph' type. Reviewers using the Browse view will then understand that fields with an = sign are for relational searches and fields with a : sign are full-text searchable. |
Understanding Field Structure and Applying Properties
Concordance Desktop does not have a pre-defined field structure and there are no required fields. However, preliminary planning in how you construct fields and apply properties to each is essential.
Concordance’s field profile includes:
•No predefined structure, no required fields
•Maximum of 250 fields per database
•Field Name maximum of 12 characters, including:
oLetters
oNumbers at middle or end
oUnderscoring in the middle only
•Field names are stored in capital letters
•Field names save when the data type is selected
|
Media (image) key field names with Unicode characters are not supported. |
Concordance Field Properties
Image |
Indicates which field contains the image key or alias, only selected once per database and is usually the BEGNO field. |
|---|---|
Media |
Indicates the field that contains the media key or alias, only selected once per database and is usually the BEGNO field. |
Key |
Speeds sorting and relational searching (adds values to database .key file), keying everything dilutes the value. |
Accession |
Copies UUID in system table into visible field, good for sorting by load order, great for tracking gaps |
System |
Field cannot be seen by users, Concordance Desktop creates these for replication/synchronization information |
Indexed |
Enables full-text searching (adds values to .ivt and .dct files) |
Guidelines for Creating Fields
Bates Numbering |
A unique serial number used to identify a record. When setting up the numbering system, think of the longest number you might need given your case record load. Designate a length that is large enough so data does not truncate. Choose text as the field’s properties in case you need an alphanumeric system implemented. |
Media field |
Select the Media (Image) Key setting to indicate the media key or alias field. You can click the View image (camera) button to launch and link the imagebase files with the corresponding Concordance Desktop database records. Only make this selection once per database for a unique field, this is generally the BEGNO field. |
Keyed fields |
Select the Key setting to improve sorting and relational searching speed. |
Indexing |
Indexing puts data into the dictionary and index. You can index any data type. Paragraph fields are indexed by default. Full-text searching only works on indexed fields. Avoid indexing unique values, serial, Bates numbers, and dates to optimize the full-text searching speed. Use relational searching for non-indexed fields. |
OCR2 field |
This field is created as an overflow field for OCR1, just in case the 12 million character limit for OCR1 is exceeded. This field name must have the same alpha prefix as the primary field and the numeric suffix must be a consistent width and start at 1. Fields must be entered in order by suffix. |
Punctuation |
Your customizable Punctuation list designates what punctuation you can use for full-text searches, as long as the characters are embedded between alphanumeric characters and are within quotes. You only need to set punctuation once for each database. |
Dictionaries/Indexes |
Big dictionary and index files slow search processing. Do not bloat these files with unnecessary entries; build stopword lists to exclude them. |
Field Validation |
Some fields require additional attributes for tracking purposes, like EDITTRAIL and CREATIONDATE date fields. These attributes can be set in the Modify window, and must be added before importing load files in order to capture the information. |
Table view |
Field column width in Table view is determined in the Types setting in the Modify dialog box. The default column width is the field length identified in the Data Types table, and as set in field properties. And by default, paragraph fields aren’t included in Table view to keep display speed optimal. You can customize Table view to include paragraph fields at any time. |
Additional Fields and Naming Conventions
Use the following table as a reference when planning your database field structure. You may find that adopting the use of these fields is beneficial to your administrative functions or provides alternative naming conventions for fields previously mentioned. Use this table to brainstorm other fields that make sense to your organization’s processes and use of Concordance Desktop.
Field |
Type |
Settings |
|---|---|---|
Serial Numbers |
Text |
Can be used to link to image files if BEGNO is not used. May have many serial #s during the life of a database, usually matched with the .tif file name as the first field in the database. |
BEGBATES BEGDOC STARTPAGE |
Text |
Alternative field names for BEGNO. |
ENDBATES ENDDOC ENDPAGE |
Text |
Alternative field names for ENDNO. |
DOCNO |
Text |
Document number, an alternative to using serial numbers key to the page, more common in e-documents that are not .tif files. |
BEGATTACH |
Paragraph |
Used to denote attachment range. |
ENDATTACH |
Paragraph |
Used to denote attachment range. |
INCLUDES |
Text |
Holds Bates or control numbers of all pages inside the document, facilitates searching for middle page .tif files. |
PARAGRAPH |
Paragraph |
Alternative name for DOCTYPE field. |
Document metadata fields |
Paragraph |
Also called bibliographic metadata fields. Typically includes author, recipient, custodian, dates (sent, received filed, etc.), subject, title, etc. Generally this information is visible on the face of the document. |
System metadata fields |
Paragraph |
Typically includes information captured automatically by the computer like last access date, modification date, and print date. This information may not appear on the face of the document. |
TEXT1 TEXT2 |
Paragraph |
Alternative naming convention for document-level text fields like OCR1, OCR2, OCR3. If these fields are named with the same alpha prefix (TEXT for example) and have numeric suffixes that start with 1, are the same numeric length, and are in order ascending by suffix, the import process overflows additional text into subsequent fields, as needed. |
BODY, TEXT or MESSAGE |
Paragraph |
Naming conventions for e-documents that are not OCR scanned, fields names are usually with numeric suffixes. |
ATTACHMENT or FILEPATH |
Paragraph |
Customarily used to hold a clickable hyperlink to the native document in electronic format. If this field contains a file path or web address, running the CreateHyperlinks.cpl converts it to a hyperlink. |
DOCCOND |
Paragraph |
Holds notes about the document condition, like Marginalia. |
PRIVCALL |
Paragraph |
Holds the reason a document or redacted sections are marked as privileged, like Attorney-Client, Priest-Penitent, Medical, etc. |
MENTIONS |
Paragraph |
Keywords included in the document. |
LOADDATE |
Date |
Alternative name for CREATEDATE field. Used to record the date the record was created, date data type. |
AUDITTRAIL |
Paragraph |
Alternative name for EDITTRAIL field. Records the date, time zone, computer session ID, and user name any time a record is changed in Edit mode, and paragraph data type. |
DISCSOURCE or SOURCE |
Text |
Used for entering the name of the disk the data is loaded from or physical media it is delivered on, includes disk number, case number, client number, name of person who loaded it, etc. |
Administrative fields with different data types |
Paragraph or Date |
Consider creating ADMIN1PARA, ADMIN1TEXT, ADMIN1DATE, etc. to hold specific types of data. |
Template for administrative fields |
(not applicable) |
Consider creating this template. Each time you make a new database, you can insert fields above them to accommodate data. To create a template: on the Documents menu, point to Export, and then click Structure. Locate and save the existing database structure in the Templates folder that is located in the same folder as the Concordance Desktop .EXE file. |
REVIEWEDBY |
Paragraph |
Alternative name for REVIEWSTATUS. Name of the attorney who reviewed the file. |
TAGSATPROD |
Paragraph |
Alternative name for PRODTAGS1. Tags as they existed for production. |
Configure Visibility of Empty Fields
You can configure a database such that empty fields are not visible in the field listing. Empty fields are those that contain no values. Hiding unused fields can help to improve the readability of database records for end users. Showing empty fields is generally a preference for Concordance Desktop administrators, who typically want to have all database fields visible whether they contain data or not.
Database with empty fields hidden
Database with empty fields shown
By default empty fields are not visible. This means that when you create a new database but before you import data, no fields are visible since none of the fields contain any data. After you import data, any field that does not contain data is not visible.
To Show Empty Database Fields
1.Open the Concordance Desktop database you want to view.
2.From the Tools menu, click Empties. A check mark next to the Empties command indicates that empty fields are visible.
|
If you have the option to view empties enabled, you may still see fields with no data on individual records. This situation occurs if data exists in that field for any other record in the case. |
To Hide Empty Database Fields
1.Open the Concordance Desktop database you want to view.
2.From the Tools menu, click Empties to remove the check mark next to the command and hide the empty fields from view.