GBDS Relational Databases

Introduction

This manual describes all GBDS relational databases tables, schemas, and information. GBDS uses a relational database to store the metadata about people, transactions, notification groups, setting overrides, and exceptions.

The document is divided into four sections that describe:

• The General tables;

• The Unsolved Latent (UL) tables;

• The Notification tables;

• The GBDS Settings tables.

General tables

The general tables are tables that retain vital information for GBDS operation, such as exceptions, people information, transaction information, and others. Those tables are described below.

gbds.people

The people table is intended to store the indexes for all people stored in GBDS database, and is described as follows:

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
pguid	varchar	255	not null	HBase stored person UGUID
deleted	tinyint	1	null	Indicates whether the candidate was deleted to be excluded when performing candidates listing operations

gbds.people_version

The people_version table is intended to store the information about the last changes in a person's register, and is described as follows:

Column	Type	Size	Value	Additional Information
version	int	11	not null	Primary key; Version index (incremental from the first change)
person_id	bigint	20	not null	Reference to person's gbds.people.id
_timestamp	datetime	6	null	Timestamp for the current version of a person
deleted	tinyint	1	null	Indicates whether the candidate was deleted to be excluded when performing candidates listing operations
active	tinyint	1	null	Defines if a version of a people is elegible for Master Record transactions

gbds.transactions_ref

The transactions table is intended to store the indexes for all transactions stored in GBDS database, and is described as follows:

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
tguid	varchar	255	not null	HBase stored transaction UGUID

gbds.transactions

The transactions table is intended to store information about quality analysis of the biometric data for a transaction.

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
tguid	varchar	255	not null	Reference to transaction's transactions_ref.tguid
pguid	varchar	255	null	HBase stored person UGUID
active	tinyint	1	not null	Defines whether a version of a person is elegible for Master Record transactions
finger_quality_extracted	tinyint	1	null	False if the fingerprints can be extracted
face_quality_extracted	tinyint	1	null	False if the face can be extracted in the background. True if already extracted
created	datetime	6	null	Transaction creation timestamp
updated	datetime	6	null	Transaction last modification timestamp
enroll_status	varchar	255	null	Status of the enroll (as ENQUEUED, PROCESSING, FAILED, and others)
quality_status	varchar	255	null	Status of the quality analysis (as OK, PENDING, APPROVED, and others)
extraction_time	varchar	255	null	Elapsed time for template extractions
extraction_quality	varchar	255	null	Elapsed time for quality extractions
match_time	int	11	null	Elapsed match time for this transaction
total_time	int	11	null	Total operation minus queue waiting time
type	varchar	20	null	Transaction type. Enum: ENROLL, UPDATE, VERIFY, IDENTIFY
fingerprint_global_quality	int	11	null	Fingerprint global quality score
global_quality	int	11	null	Profile global quality score
quality_extraction_api_id	varchar	255	null	API id that will perform or performed quality extraction
latent	tinyint	1	null	Flags whether the transaction is a latent search
ul	tinyint	1	null	Flags whether the transaction is a UL search
api_id	varchar	255	null	The ID of the API instance that received the transaction
num_fingers	int	11	null	The number of fingerprints in the transaction
num_faces	int	11	null	The number of face images in the transaction (0 or 1)
gbds_version	varchar	255	not null	GBDS version that processed the transaction
ginger_extractor_type	enum	n/a	null	Type of ginger extractor used for the transaction. Enum: GRIAULE_FAST, GRIAULE_BASIC, GRIAULE_2020, GRIAULE_2024, GRIAULE_2018.

gbds.transaction_fields

The transaction_fields table is intended to store the information about the quality information and link it with the non-biometric information of the fields table.

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
fkey	varchar	255	null	Field key describing the contained information (e.g. whether it is a blood type, mother's name, birth date, document, etc) Reference to fields.fkey
type	varchar	255	null	Type of the information (search or label) Reference to fields.type
fvalue	varchar	255	null	Field effective value (e.g. the document number) Reference to fields.fvalue
transaction_id	bigint	20	not null	Id of the transaction. Reference to transaction.id

gbds.transaction_fingerprint_quality

Column	Type	Size	Value
transaction_id	bigint	20	not null
idx	int	11	not null
image_quality	int	11	null
template_quality	int	11	null
minutiae_count	int	11	null
blank	varchar	100	null
contrast	varchar	100	null
fingerArea	varchar	100	null
fingerCenterX	varchar	100	null
fingerCenterY	varchar	100	null
fingerExtentX	varchar	100	null
fingerExtentY	varchar	100	null
hasjoint	varchar	100	null
__index	varchar	100	null
marginBiteEast	varchar	100	null
marginBiteNorth	varchar	100	null
marginBiteSouth	varchar	100	null
marginBiteWest	varchar	100	null
nfiq	varchar	100	null
orientation	varchar	100	null
sizeX	varchar	100	null
sizeY	varchar	100	null

gbds.transaction_face_quality

Column	Type	Size	Value
transaction_id	bigint	20	not null
idx	int	11	not null
image_quality	int	11	null
template_quality	int	11	null
autoBrightness	varchar	100	null
bgBelowPictureQuality	varchar	100	null
bgBlueStandardDeviation	varchar	100	null
bgDarknessQuality	varchar	100	null
bgGreenStandardDeviation	varchar	100	null
bgRedStandardDeviation	varchar	100	null
bgUniformityQuality	varchar	100	null
blurCompliance	varchar	100	null
busyBackground	varchar	100	null
busyBackgroundInCropped	varchar	100	null
cropContainmentError	varchar	100	null
eyesLookDown	varchar	100	null
eyesLookLeft	varchar	100	null
eyesLookRight	varchar	100	null
eyesLookUp	varchar	100	null
eyesObstruction	varchar	100	null
eyesTooClosed	varchar	100	null
eyesTooOpen	varchar	100	null
faceDown	varchar	100	null
faceLeft	varchar	100	null
faceObstruction	varchar	100	null
faceOrientationPitchCompliance	varchar	100	null
faceOrientationRollAngle	varchar	100	null
faceOrientationYawCompliance	varchar	100	null
faceRight	varchar	100	null
faceUp	varchar	100	null
glasses	varchar	100	null
grayscaleSpan	varchar	100	null
hat	varchar	100	null
heavyGlasses	varchar	100	null
icaoCompliance	varchar	100	null
leftEyeX	varchar	100	null
leftEyeY	varchar	100	null
mouthObstruction	varchar	100	null
mouthOpen	varchar	100	null
obstruction	varchar	100	null
openMouth	varchar	100	null
pictureHeight	varchar	100	null
pictureWidth	varchar	100	null
pixelated	varchar	100	null
redEye	varchar	100	null
result	varchar	100	null
rightEyeX	varchar	100	null
rightEyeY	varchar	100	null
saturated	varchar	100	null
saturationGrayscaleDistribGrade	varchar	100	null
saturationNumGrayTones	varchar	100	null
saturationOverExposure	varchar	100	null
shadows	varchar	100	null
skinColorCompliance	varchar	100	null
smile	varchar	100	null
smilingMouth	varchar	100	null
spoof	varchar	100	null
spoofGrade	varchar	100	null
tiltAngle	varchar	100	null
tooDark	varchar	100	null
unnaturalSkinColor	varchar	100	null
visibleTeeth	varchar	100	null
wrongFacePose	varchar	100	null
wrongShoulderPoseLeft	varchar	100	null
wrongShoulderPoseRight	varchar	100	null

gbds.biometrics

The biometrics table is intended to store the biometrics of a person, being linked to the transaction and the people_version tables. It is described as follows:

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
idx	int	11	null	Index of the current biometric
quality	int	11	null	Quality score for the extracted biometric template
type	varchar	255	null	Modality of the current biometric
person_id	bigint	20	not null	Reference to person's gbds.people.id
transaction_id	bigint	20	not null	Reference to transaction's transactions_ref.id
person_version	bigint	11	not null	Reference to person's people_version.version

gbds.fields

The fields table is intended to store the non-biometric information of a person, being linked to the transaction and the people_version tables. It is described as follows:

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
fkey	varchar	255	null	Field key describing the contained information (e.g. whether it is a blood type, mother's name, birth date, document, etc)
type	varchar	255	null	Type of the information (search or label)
fvalue	varchar	255	null	Field effective value (e.g. the document number)
person_id	bigint	20	not null	Reference to person's person.id
person_version	int	11	not null	Reference to person's people_version.version

gbds.exceptions

The exceptions table is intended to store the information of any biometric exception and its treatment. It is described as follows:

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary key; Relational database private key
aguid	varchar	255	not null	HBase stored exception UGUID
comments	varchar	255	null	Any comments given when treating the exception
_timestamp	datetime	6	null	Timestamp for the creation of the exception
status	varchar	255	not null	Current status of the exception defined whether it is treated or pending
user	varchar	255	null	Identification of the user responsible for treating the exception
reference_person_id	bigint	20	not null	Foreign key: Reference to reference person's people.id
entrant_person_id	bigint	20	not null	Foreign key: Reference to entrant person's people.id
transaction_id	bigint	20	not null	Foreign key: Reference to transaction's transactions_ref.id
reference_person_version	int	11	not null	Foreign key: Reference to reference person's people_version.version
entrant_person_version	int	11	not null	Foreign key: Reference to entrant person's people_version.version

gbds.status

The status table is intended to store the information about nodes and matchers of the GBDS.

Column	Type	Size	Value	Additional Information
guid	varchar	255	not null	Primary Key; Task ID
stimestamp	datetime	6	null	Timestamp for status timeout
people_count	bigint	20	null	Number of people in a node or cluster
ul_count	bigint	20	null	Number of UL in a node or cluster
info	longblob	Up to 4Gb	null	Status information and data

gbds.apis

The apis is used by GBDS to manage multiple APIs active at the same time.

Column	Type	Size	Value	Additional Information
api-id	varchar	255	not null	ID of the unique API instance
hostname	varchar	255	not null	Hostname of the node where the API is running
port	int	11	not null	Port where the API is running
type	varchar	255	null	Type of the API instance (LEADER, RUNNER, or null)

On API boot, each API will look for itself in this table. If not found with hostname/IP and port, it inserts itself in the table with a GUID as api-id and type null, meaning not ready for quality extraction. Changes directly applied to this table will be parsed every 15 minutes.

Unsolved Latent (UL) tables

The Unsolved Latent tables are used to store information about the UL information and candidates. The tables are described below.

gbds.ul

The ul table is designed to store all the GBDS UL's and is described as follows:

Column	Type	Size	Value	Additional Information
uguid	varchar	255	not null	Primary key; HBase stored UGUID
ul_status	varchar	255	not null	UNSOLVED or SOLVED; HBase replicates this status
creation_time	timestamp	4	not null	current_timestamp by default
person_pguid	varchar	255	null	PGUID of the matched person when SOLVED
person_tguid	varchar	255	null	TGUID of the matched person when SOLVED
fragment_id	varchar	255	null	Original fragment ID for the fragment that generated the UL
fragment_case_id	varchar	255	null	Original case ID for the fragment that generated the UL
fragment_index	int	11	null	Fragment index for the UL. By default, the index is set as -1 (index not known, any index)
analysis_user	varchar	255	null	User responsible for the UL analysis
analysis_timestamp	timestamp	4	null	Analysis timestamp
group_guid	varchar	255	not null	Grouping GUID (for listing linked UL's)

gbds.ul_candidate

The ul_candidates table is designed to store the candidates of each UL that GBDS keeps after any REVERSE LATENT MATCH done over enrollments that generated a match against a UL, and is described as follows:

Column	Type	Size	Value	Additional Information
ul_uguid	varchar	255	not null	Primary key; UL UGUID
person_pguid	varchar	255	not null	Primary key; Candidate's PGUID
person_tguid	varchar	255	not null	Primary key; Candidate's TGUID
person_index	int	11	not null	Primary key; Candidate finger index
score	int	11	not null	Matching score for the candidate
deleted	tinyint	1	not null	0 by default, indicates whether the candidate was deleted to be excluded when performing candidates listing operations
minutiae	longblob	Up to 4Gb	null	JSON serialization containing the matched minutiae. It's a list from the following structure: - queryIndex, int - referenceIndex, int

Notification tables

The notification tables are used to store data for auditorship purposes, as e-mails, persons who will be notified, and groups.

gbds.notify_user

The notify_user table is designed to store the data of the gbds authenticated user.

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary Key; User unique ID.
username	varchar	255	not null	authenticated gbds username

gbds.notify_group

The notify_group table is designed to store the notification groups information.

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary Key;
name	varchar	255	not null	Group Name
enabled	tinyint	1	not null	Defines if the group will be active or not

gbds.notify_group_email

The notify_group_email table is designed to store the emails of a given group.

Column	Type	Size	Value	Additional Information
notify_group_id	bigint	20	not null	Primary Key; ID of the group.
email	varchar	255	not null	Primary Key; Email that belongs to the group

gbds.notify_user_group

The notify_user_group table is designed to store the information of which group a given user is part of.

Column	Type	Size	Value	Additional Information
notify_user_id	bigint	20	not null	Primary Key; Reference to user's notify_user.id
notify_group_id	bigint	20	not null	Primary Key; Reference to group's notify_group_email.notify_group.id

gbds.people_transparency

The people_transparency table is designed to store the information about a given person and what actions are taken when this person is searched.

Column	Type	Size	Value	Additional Information
id	bigint	20	not null	Primary Key;
pguid	varchar	255	not null	Reference to person's people.pguid
enabled	tinyint	1	not null	Indicates if the action is on or off for the given person
action	varchar	255	null	Action to be taken

gbds.people_transparency_group

The people_transparency_group table is designed to store the information about which groups a person belongs to.

Column	Type	Size	Value	Additional Information
people_transparency_id	bigint	20	not null	Primary Key; Reference to people_transparency's people_transparency.id
notify_group_id	bigint	20	not null	Primary Key; Reference to group's notify_group_email.notify_group.id

GBDS Settings tables

GBDS uses a relational database to store some settings for the API and GBDS. The settings table is a special table designed to control certain GBDS and GBDS API configurations. These configurations are stored in the gbds.settings table. The table can contain configurations present in GBDS API, GBDS, or both following the schema below:

Type	Type	Size	Value	Additional Information
skey	varchar	255	not null	Primary key. Parameter key name.
stype	varchar	50	not null	Local where is the parameter from. API or GBDS
svalue	varchar	4096	null	Value of the parameter
description	varchar	4096	null	Description of the parameter
stimestamp	datetime	6	not null	Timestamp
host	varchar	1024	null	Hostname of a node

All settings found inside the table will be written in the respective file, gbdsapi.properties for GBDS API and application.conf for GBDS, every 15 minutes. Also, it will propagate all updated configurations on memory in both, API and GBDS.

The parameter value defined at gbds.settings table will propagate to ALL nodes.

This feature is controlled by one configuration parameter in the table, gbds.rdbSystemConfiguration.enabled. This parameter will enable the overwrite of the values of GBDS and API configuration. Setting the parameter value to BOTH will make GBDS and API configurations be overwritten.

gbds.settings / File Configuration Key	type
gbscluster.min.quality	API
gbds.enroll.fingerprints.min-nr-template	API
gbds.enroll.face.min-nr-template	API
gbds.enroll.iris.min-nr-template	API
gbds.enroll.palmprint.min-nr-template	API
gbds.enroll.newborn-palmprint.min-nr-template	API
gbscluster.enroll.fingerprints.verify.matchthreshold	API
gbscluster.update.min.quality	API
gbds.transparency.search.identify.request.notify.enabled	API
gbds.transparency.search.identify.result.actions.enabled	API
gbds.api.logLevel	API
gbds.extraction.service	API
gbds.extraction.service.face.count	API
gbds.extraction.service.ginger.count	API
gbds.extraction.service.girl.count	API
gbds.extraction.service.hostname	API
gbds.extraction.service.initialPort	API
gbds.extraction.service.logLevel	API
gbds.extraction.service.maxTries	API
gbds.extraction.service.linkLibSegfault	API
gbds.extraction.quality.service	API
gbds.extraction.quality.fillTransactionQualityPropertiesTable	API
gbds.faces.extraction.quality.api	API
gbds.faces.extraction.quality.background	API
gbds.fingerprints.extraction.quality.api	API
gbds.fingerprints.extraction.quality.background	API
gbds.extraction.quality.service.finger.count	API
gbds.extraction.quality.service.face.count	API
gbds.extraction.quality.service.initialPort	API
gbds.extraction.quality.service.logLevel	API
gbds.extraction.quality.service.timeout	API
gbds.extraction.quality.service.hostname	API
gbds.extraction.quality.service.maxTries	API
gbds.extraction.quality.service.linkLibSegfault	API
gbds.extraction.quality.service.rows-on-select	API
gbds.extraction.quality.service.submitted-queue-factor	API
gbds.enroll.face.min.quality	API
gbds.update.face.min.quality	API
gbds.monitor.url	API
gbds.template.face.multiplicity	API
gbds.biographicBase.enabled	API
gbds.biographicBase.endpoints	API
gbds.biographicBase.get.timeout.ms	API
gbds.biographicBase.list.timeout.ms	API
gbds.biographicBase.logLevel	API
gbds.biographicBase.clientID	API
gbds.biographicBase.clientSecret	API
gbds.biographicBase.lookAllServers	API
gbscluster.fingerprints.extraction.enroll.type	API
gbscluster.fingerprints.extraction.verify.type	API
gbds.update.exception.reextract	API
gbds.update.exception.reextract.save	API
gbds.biographicBase.autoUpdate	API
gbds.biographicBase.sendPguidAsKey	API
gbds.biographicBase.sendTguidAsKey	API
gbds.log.diagnose	GBDS
gbds.ul.boot.scan.enabled	GBDS
gbds.boot.scan.ignoreErrorsOnRegion	GBDS
gbds.boot.matcher.creation.sleepTime.ms	GBDS
gbds.biometric.fingerprint.identify.threshold	GBDS
gbds.biometric.fingerprint.exception.threshold	GBDS
gbds.biometric.fingerprint.exception.enabled	GBDS
gbds.biometric.fingerprint.exception.enroll.min-matches-for-exception	GBDS
gbds.biometric.face.identify.threshold	GBDS
gbds.biometric.face.exception.threshold	GBDS
gbds.peopleList.countFromRDB	GBDS
gbds.biometric.face.enabled.threshold	GBDS
gbds.driver.logLevel	GBDS
gbds.log.loadUnload	GBDS
gbds.template.memory.format	GBDS
gbds.match.service.enabled	GBDS
gbds.match.service.initialPort	GBDS
gbds.match.service.logLevel	GBDS
gbds.match.service.timeout	GBDS
gbds.match.service.templateSend.parallelByModality	GBDS
gbds.match.service.linkLibSegfault	GBDS
gbds.match.service.maxTries	GBDS
gbds.match.service.maxConnectionErrors	GBDS
gbds.memory-monitor	GBDS
gbds.watchdog.interval	GBDS
gbds.watchdog.log.mode	GBDS
gbds.watchdog.log.level	GBDS
gbds.verifyPostMatch.enabled	GBDS
gbds.transparency.search.identify.result.notify.enabled	BOTH
gbds.transparency.email-notifier.url	BOTH
gbds.transparency.email-notifier.log-level	BOTH
gbds.transparency.email-notifier.timeout	BOTH
gbscluster.update.consider.fingerprints	BOTH
gbscluster.update.consider.faces	BOTH
gbscluster.update.consider.faces.beforeFingerprints	BOTH
gbscluster.update.faces.verify.matchthreshold	BOTH
gbscluster.update.minimum.fingers	BOTH
gbds.search.verify.adjust-resolution	BOTH

Other configurations can be placed on table gbds.settings on rdb and all will be written on API or GBDS file according to configuration type. But the memory reloads on execution time will not be performed for these new configurations, only after API and/or GBDS are restarted.