Types of Data

Understanding Types of Data

When a support team collects customer feedback, it might store ticket numbers in a table, receive nested JSON messages, and keep customers’ written descriptions. These illustrate structured, semi-structured, and unstructured data. The distinction concerns how information is organized for access and analysis, not whether it is useful, accurate, or digital. One project can contain all three forms simultaneously.

Structured data follows a defined arrangement of fields, commonly rows and columns. In a ticket table, each row might represent one ticket, with columns for ticket_id, priority, and response_minutes. A schema describes expected fields and their types or constraints. This consistency makes filtering, joining, and aggregation straightforward. However, a rectangular layout does not guarantee valid values: negative response times or duplicated ticket identifiers still require checks.

Semi-structured data carries organizational clues such as keys, tags, and nesting without necessarily following one flat table layout. JSON messages can contain a customer object, a list of events, and optional fields. Programs can parse that organization, but analysts must decide how to handle absent keys and repeated elements. JSON can also follow a strict schema; its file format alone does not establish how consistent the records are.

Unstructured data does not present its analytical meaning as predefined fields. A customer’s paragraph, a photograph, or a recording requires interpretation or feature extraction before many analyses are possible. “Unstructured” does not mean structureless: images have pixel arrays, and language has grammar. The distinction is about the information being analyzed. Putting a paragraph into a database column does not turn its meaning into neatly separated variables.

A separate classification describes what individual values mean. Categories such as department are nominal; ordered labels such as low, medium, and high are ordinal. Counts are discrete numerical values, while measurements such as duration are often treated as continuous. An identifier containing digits is usually a label, not a quantity to average. These distinctions guide valid comparisons and summaries. The pandas user guide treats tabular structures, text, categories, and missing values separately, reflecting these different concerns. No representation is universally best: tables simplify repeated calculations, nested records preserve relationships, and original media retain information that extracted features may lose.

Practical walkthrough

Use Python 3.10 or later; the example needs only the standard library and basic familiarity with dictionaries, lists, and loops. Everything stays in memory. The task is to turn two nested support-ticket records into consistent rows while retaining their original messages.

First, establish the unit of observation: one row represents one ticket, not one customer or one word. This decision matters because a customer could submit several tickets. Counting rows would then count tickets, not distinct customers. We assume the supplied ticket identifiers are unique and each record contains a customer object and a text message.

Next, inspect the JSON organization. The customer’s tier sits inside a nested object, while priority is optional. Parsing with json.loads exposes these fields as Python dictionaries and lists. Parsing checks JSON syntax, but it does not verify business rules such as valid priority labels or unique identifiers.

Build a new dictionary for each ticket using the same field names. Extract tier from the nested customer object. Use get for priority so that an absent value becomes None. Do not silently replace a missing priority with low: absence means the source did not provide a value, not that someone assigned the lowest urgency.

Finally, derive message_words using whitespace splitting and print selected fields. This creates a discrete numerical feature from free text, but it does not measure satisfaction or urgency. Punctuation, languages without spaces, and alternative tokenization rules limit its usefulness. Keep the original message alongside the count so later analysis can revisit this deliberately simple transformation.

Worked example

import json

payload = '''[
  {"ticket_id": "T1", "customer": {"tier": "basic"},
   "priority": "high", "message": "Package arrived very late"},
  {"ticket_id": "T2", "customer": {"tier": "plus"},
   "message": "Thanks for your help"}
]'''

records = json.loads(payload)
rows = []
for record in records:
    message = record["message"]
    rows.append({
        "ticket_id": record["ticket_id"],
        "tier": record["customer"]["tier"],
        "priority": record.get("priority"),
        "message": message,
        "message_words": len(message.split()),
    })

print("ticket_id | tier | priority | message_words")
for row in rows:
    priority = row["priority"]
    if priority is None:
        priority = "(missing)"
    print(f"{row['ticket_id']} | {row['tier']} | {priority} | {row['message_words']}")

Expected output

ticket_id | tier | priority | message_words
T1 | basic | high | 4
T2 | plus | (missing) | 4

The output preserves the order of the two records in the JSON array. Both messages contain four whitespace-separated words, so both counts are 4 despite their different meanings. T1 supplies high explicitly; T2 lacks the priority key, so its stored None appears as (missing). The original nested records are semi-structured. The resulting dictionaries share a tabular schema, although their message field still contains unstructured text. ticket_id is an identifier, tier is categorical, and message_words is numerical. This example performs representation changes only: it neither trains a model nor learns anything about the relationship between word count and customer experience.

Common mistakes and how to avoid them

• Classifying an entire dataset solely by its container.

  Inspect both the outer organization and each field’s contents. A SQL table can hold free text; a JSON record can contain consistently typed measurements.

• Treating numeric-looking identifiers as measurements.

  Ask whether arithmetic has meaning. Preserve identifiers such as postal codes as labels, including leading zeros, rather than averaging them or interpreting larger codes as greater quantities.

• Flattening repeated elements without checking row counts.

  If a ticket contains several events, choose a separate event table or an explicit aggregation. Expanding events into rows can duplicate ticket-level values and inflate totals.

• Confusing successful parsing with validated data.

  After parsing, check required keys, value types, allowed categories, and identifier uniqueness. Define whether missing fields should remain missing, trigger rejection, or receive a documented default.

Review questions

Further reading

• pandas user guide
• scikit-learn user guide