Importing and Understanding Research Data

DRS Training

💡 Learning Outcomes

  • Choose an object structure and inspect its dimensions and types.
  • Import a table without changing the source or losing its header.
  • Identify observation level, keys, units and recorded years.
  • Retain missing values and trace a selected record back to its source.

❓ Questions

  1. What does one row represent?
  2. Which fields identify an observation and which are measurements?
  3. What changes when a value is missing or a column has the wrong type?

Structure & Agenda

Block Teaching Activity Output to keep
Structures and columns 10 min 5 min An object choice
Read and inspect 10 min 5 min An import record
Identifiers and units 10 min 5 min A source check
Break — 10 min break —
Select observations 10 min 5 min A checked subset
Types and missingness 10 min 5 min A missing-value record
Save the import workflow 10 min 5 min A rerunnable input audit
Total 60 min 30 min 100 min including the break

This is lesson 2 of eight. Each five-minute activity includes a room response and debrief. Save work before the ten-minute break; use a practice copy and keep original inputs unchanged.

Starting Point and Preparation

Complete lesson 1, open the course project and download data/gapminder_data.csv using Setup. The file is historical country-year data, not individual measurements.

Locate the input, inspect its types and observation level, check identifiers and units, then retain an audit record with the selected data.

Check the input before analysing it.

Choose a Structure for the Information

Lists Preserve Different Kinds of Information

List of 4
 $ : chr "a"
 $ : int 1
 $ : num 3.14
 $ : chr [1:7] "C" "D" "E" "F" ...
[[1]]
[1] "C" "D" "E" "F" "G" "H" "I"
[1] "C" "D" "E" "F" "G" "H" "I"

A list can contain different types and nested objects. stuff[4] returns a one-element list; stuff[[4]] returns the element inside it. Distinguish those results before passing them to a function.

Matrices Have Two Dimensions and One Type

[1] 6
c d e 
4 5 6 
a b c 
3 6 9 

matrix() accepts data, nrow, ncol, byrow and optional dimension names. A matrix stores one type of value. Use [row, column]; omitting one index selects the whole row or column. Check dimensions before applying a calculation along an axis.

Data Frames Keep Columns with Different Types

'data.frame':   2 obs. of  3 variables:
 $ sample_id: chr  "sample_01" "sample_02"
 $ mass_g   : num  2.1 2.4
 $ checked  : logi  TRUE FALSE
  sample_id mass_g checked
1 sample_01    2.1    TRUE
[1] 2.1 2.4

This small example is fictional. Columns can have different types, while each column has a consistent structure. A tibble is a data-frame variant used in the tidyverse. A data-frame row can contain both text and measurements: do not blindly convert it to numeric values or average every column.

Task 1: Choose an Object and Retain Identity (5 min)

Goal: distinguish a list, matrix and data frame Input: stuff, mx and the fictional samples table Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: Predict the difference between stuff[4] and stuff[[4]].
  2. 2 min: Inspect the objects and select the sample_id and mass_g columns.
  3. 1 min: Check retained IDs, order and measurement units.
  4. 1 min: Two pairs explain why averaging a whole mixed-type row would be inappropriate.

Keep: the selected table and object explanation. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

A list can hold unlike objects; a matrix has one type; a data frame preserves text identifiers beside measurements. The structure affects valid operations.

Import a Table without Editing Its Source

Read an Input without Changing It

The downloaded CSV contains historical country–year records from Gapminder. Read it into a named object rather than editing the input file. read.csv() uses a header row by default; checking the column names confirms that the header was interpreted as intended.

      country year      pop continent lifeExp gdpPercap
1 Afghanistan 1952  8425333      Asia  28.801  779.4453
2 Afghanistan 1957  9240934      Asia  30.332  820.8530
3 Afghanistan 1962 10267083      Asia  31.997  853.1007
4 Afghanistan 1967 11537966      Asia  34.020  836.1971
5 Afghanistan 1972 13079460      Asia  36.088  739.9811
6 Afghanistan 1977 14880372      Asia  38.438  786.1134
[1] "country"   "year"      "pop"       "continent" "lifeExp"   "gdpPercap"
[1] 1704    6
'data.frame':   1704 obs. of  6 variables:
 $ country  : chr  "Afghanistan" "Afghanistan" "Afghanistan" "Afghanistan" ...
 $ year     : int  1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 ...
 $ pop      : num  8425333 9240934 10267083 11537966 13079460 ...
 $ continent: chr  "Asia" "Asia" "Asia" "Asia" ...
 $ lifeExp  : num  28.8 30.3 32 34 36.1 ...
 $ gdpPercap: num  779 821 853 836 740 ...

The explicit stringsAsFactors = FALSE retains text columns as character values for this import. Inspect the actual types rather than assuming that the CSV retains an R object’s metadata.

Check the Header and Types

A CSV stores text, not all the metadata of an R object. Compare names, dimensions and a source row. If importing Excel later, record the sheet, range, header and missing-value conventions; the file extension alone cannot establish them.

Task 2: Inspect an Import (5 min)

Goal: verify an imported table before selecting records Input: the original Gapminder CSV and read.csv() example Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: Identify the file path and expected header.
  2. 2 min: Read the table and inspect names, dimensions and types.
  3. 1 min: Compare the first observation with the source CSV; retain its identifiers.
  4. 1 min: Two pairs explain a check that catches a header or type problem.

Keep: an input record and saved import code. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

The first observation must remain data rather than become a column name. A readable file still needs observation-level and type checks.

Check Observation Level, Identifiers and Units

[1] 1952 2007
[1] 0
      country year lifeExp
1 Afghanistan 1952  28.801

The country–year combination identifies an observation. Check the available years before answering a question about a particular date. Life expectancy is measured in years and population in people; confirm definitions in the Gapminder documentation.

Room prediction: ask whether two rows for the same country in different years are duplicates. The country alone repeats; the combination of country and year represents distinct observations.

Task 3: Explain the Key and Units (5 min)

Goal: define what makes an observation distinct Input: gapminder_csv and its data documentation Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: Predict whether country alone identifies each row.
  2. 2 min: Check country-year uniqueness and the recorded year range.
  3. 1 min: Record units for life expectancy and population.
  4. 1 min: Two pairs explain why two years for one country are distinct observations.

Keep: the key, source coverage and units. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

Country-year identifies these records. Available years and aggregate measurement units constrain the questions this source can answer.

Break (10 min)

Save your import record and script.

Select Observations and Keep Their Keys

Make the Condition Visible

    country year lifeExp       pop
180  Brazil 2007  72.390 190010647
756 Ireland 2007  78.885   4109086

%in% tests membership; & combines conditions for each row. Keep IDs alongside measurements and compare retained values with their original records. An empty selection is a cue to investigate coverage or spelling, not substitute observations silently.

Task 4: Check a Selected Comparison (5 min)

Goal: select exactly the intended country-year records Input: the two-country selection above Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: State the intended countries and year before running the code.
  2. 2 min: Select the rows and retain identifiers and units.
  3. 1 min: Compare both observations with the original table.
  4. 1 min: Two pairs explain what an empty selection would require them to check.

Keep: a checked subset and its selection rule. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

A correct row count is helpful but insufficient: identifiers, year and values must also agree with the intended source records.

Missing Values and Type Decisions

Preserve the Uncertainty

The following is a fictional altered copy, used to practise missingness; it does not change the source data.

[1] FALSE  TRUE FALSE
[1] NA
[1] 30.399

The first mean is missing. The second uses only observed values and needs its denominator recorded. That is a stated calculation choice, not evidence that replacing or omitting missing data is appropriate for a real study. Do not turn a category into numbers merely to make a function accept it.

Task 5: Explain a Missing Result (5 min)

Goal: record a missing-value choice explicitly Input: the fictional altered copy Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: Predict the two means and count the observed values.
  2. 2 min: Run the examples and inspect the retained country-year rows.
  3. 1 min: Write a sentence identifying the denominator and what remains missing.
  4. 1 min: Two pairs explain why replacing the missing value with zero would change the question.

Keep: the missing-value record and checked counts. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

The source cannot supply an invented measurement. Describe what the chosen calculation includes and retain the missing observation for review.

Save the Import and Audit

Assemble the Workflow

Save the path, import, structure checks, selection and source comparison in one script. Rerun it from a fresh session with the project open. Keep the fictional missingness example separate from the original source object.

Task 6: Rerun an Input Audit (5 min)

Goal: make an import and selection repeatable Input: the saved script and unchanged CSV Work in pairs: one operates and one checks; swap roles between tasks.

  1. 1 min: Save the input path, import, checks and selection together.
  2. 2 min: Restart R and source the script with the project open.
  3. 1 min: The checker locates a selected observation in the original file.
  4. 1 min: Two pairs identify which checks establish execution and which establish the intended selection.

Keep: the input audit, script and unresolved questions. If access fails: annotate the instructor’s displayed input and output with the same decisions and checks.

Fresh execution checks dependencies; a source-record comparison checks the selection. Neither establishes a causal or present-day interpretation.

Further Information

📚 Key Points

  • Retain the question, source, units and decisions with the result.
  • Compare at least one output with an independent known value or source record.
  • A successful run is one check; interpretation and limitations still need review.

🔦 Hints

  • Save before restarting R and rerun from the beginning.
  • Keep original inputs separate from derived outputs.
  • Record unresolved questions instead of silently changing data to obtain an answer.

Module Summary

You can inspect structures, import a table and preserve keys, units and missingness. Use those input expectations when writing functions in lesson 3.

Additional Learning

Course Overview · Previous · Next

Feedback and Register

Scan the QR code to give feedback on Digital Skills training.