# Dataset description
\iffalse
data from MicroSS (data static)
metric (number) log (number) business log (from time to time)
There are two folders named MicroSS and Company in the downloaded file.
data static for other data from our companions: metric
metric (number)
\fi
GAIA contains the data from MicroSS (in MicroSS.zip ) and metrics from companions (in Companion Data.zip ). Statistically, the data from MicroSS contains {} metrics, {} log items and detailed trace data continuously collected for weeks. In this scenario, we also simulate the anomalies that may happen in real systems and provide the record for all anomaly injections for fair evaluation of root cause analysis algorithms. This is achieved by controlling the users' behaviors and mimicking the wrong manipulations to the systems.
The data files are listed below.
| File name | Relevant repository | Download |
|---|---|---|
| MicroSS.zip | metric | trace |
| Companion Data.zip | metric | log |
MicroSS.zip
MicroSS.zip contains all data in different types, selected from the business simulation system MicroSS . It comes from a scenario of logging-in with QR Code. The description of this scenario is also included in MicroSS.zip.
>> metric
In "metric" folder, each file contains the metrics of a single node, reformulated from the raw data collected by Metricbeat. The data includes fields as follows.
| timestamp | host_ip | service_name | metric | value |
|---|---|---|---|---|
| 2021-07-01 18:00:12 | 10.1.1.115 | redisservice2 | docker_network_inbound_bytes | 34201179 |
timestamp: string of time record with the form YYYY-MM-DD hh:mm:ss
host_ip: the IP of the host running the service named service_name
service_name: name of service or host
metric: name of metric
value: value of metric at the timestamp
>> trace
In "trace" folder, each file contains the trace record, reformulated from the raw data collected by OpenTracing. The data includes fields as follows.
| timestamp | host_ip | service_name | trace_id | span_id | parent_id | start_time | end_time | url | status_code | message |
|---|---|---|---|---|---|---|---|---|---|---|
| 2021-07-01 10:54:23 | 10.1.1.117 | dbservice1 | c124e30fb40651dc | 58ac80ceea500f66 | 8b3e4a4003c5119c | 2021-07-01 10:54:22.632751 | 2021-07-01 10:54:23.151922 | http://10.1.1.117:9388/db_login_methods?uuid=a3036736-da17-11eb-9811-0242ac110003&user_id=ToeLCkHR (opens new window) | 200 | request call function 1 dbservice1.db_login_methods |
timestamp: string of time record with the form YYYY-MM-DD hh:mm:ss
host_ip: the IP of the host running the service named service_name
service_name: name of service or host
trace_id: UUID of the business trace
span_id: UUID of the node in current trace
parent_id: UUID of the parent node in current trace
start_time:the time this call is created
end_time: the time this call is closed
url: the RPC url
status_code: 200 for normal, and others for anomalies.
message: the out-band message for this call
>> business
In "business" folder, each file contains the business log of a node, reformulated from the raw data. The data includes fields as follows.
| datetime | service | message |
|---|---|---|
| 2021-07-01 00:00:00 | dbservice2 | 2021-07-01 14:11:54,950 |
datetime: string of time record with the form YYYY-MM-DD hh:mm:ss
service: the relevant node ID
message: extra information in this log.
>> run
In "run" folder, we provide system log and anomaly injection records. The data includes fields as follows, with the same meaning to files in "business" folder.
| datetime | service | message |
|---|---|---|
| 2021-07-01 | dbservice1 | 2021-07-01 22:33:05,033 |
Companion Data.zip
Companion Data.zip contains metric and log data provided by the companions of Cloudwise. All the data in Companion Data.zip has achieved strict hyposensitization to protect users and companies' privacy.
>> metric
In "metric" folder, each file records a single time series. Notice that all metrics here are labeled, so that metric anomaly detection can be tackled with fair evaluation. The data includes fields as follows.
| timestamp | value | label |
|---|---|---|
| 1546272000000 | 168899765 | 0 |
| 1546272300000 | 168900938.6 | 0 |
| 1546272600000 | 168902112.2 | 0 |
| 1546272900000 | 168896334 | 0 |
| 1546273200000 | 168880129 | 0 |
| 1546273500000 | 168863924 | 0 |
timestamp: the time of data collection.
value: metric value at the time.
label: anomaly label. 0 for normal, and 1 for anomaly.
>> log
In "log" folder, each file contains a log dataset, mainly for named entity recognition (NER) task. There are currently six data sets available: apache, hadoop, hdfs, kafka, spark, openstack. The data set is labeled with nine entities, which are date, time, message level (LVL), path, function name (FUNC), class name (CLS), HOST, IP, URL.
A log in datasets is first segmented according to spaces and some special punctuation marks. Then they are labeled in so-called IOB2 format. Each token in a log is labeled with either ‘O’ (other), ‘B-XXXX’ (beginning token for entity of type ‘XXXX’), ‘I-XXXX’ (inside token for entity type ‘XXXX’).