Iteration and Repeatable Workflows in R

DRS Training

💡 Learning Outcomes

• Trace a for loop and distinguish the current value from its position.
• Retain named results and choose a loop, vectorised operation or reusable function.
• Diagnose incomplete outputs and record missing inputs explicitly.
• Repeat a checked operation across Gapminder groups without changing the source.
• Export, read back and document derived files so a collaborator can review a run.

❓ Questions

1. When does a repeated task need a loop, and when will a vectorised operation do?
2. What changes on each iteration and what should remain unchanged?
3. How will we notice a missing output, wrong selection or unexpected overwrite?

Structure & Agenda

Block	Teaching	Activity	Output to keep
Loop values and positions	10 min	5 min	A predicted and checked trace
Retain results and compare approaches	10 min	5 min	Named numerical results and a reasoned choice
Conditions and loop diagnosis	10 min	5 min	A missing-input record and a diagnostic check
Break	—	10 min break	—
Repeat operations over groups and lists	10 min	5 min	Checked continent tables and row counts
Plan and write separate file outputs	10 min	5 min	A fresh run folder and output record
Read back, rerun and hand over	10 min	5 min	A checked export record and saved script
Total	60 min	30 min	100 min including the break

This is lesson 4 of eight, with the same 100-minute plan used throughout the course: 60 minutes teaching, 30 minutes activities and a separate 10-minute break. Debriefs are included in activity time.

Starting Point and Preparation

• Starting point: use assignment, indexing and the functions introduced in R Foundations and Functions.
• Tools and materials: open the course project and save a script named iteration.R. Use gapminder, dplyr and readr, prepared in Setup; no additional packages are needed.
• Working arrangement: pairs, with one operator and one checker; swap roles between tasks.
• Access fallback: share a screen or annotate the instructor’s code and output. Retain the proposed selections and checks even when you cannot execute them.
• File outputs: use a new folder for each practice run, separate from input data. Save your script and existing work before restarting R.

Trace a Repeated Task

A Research Task with Several Outputs

A collaborator needs one historical country-level table for each continent in the Gapminder teaching dataset. Repeating nearly identical filtering and exporting commands is easy to get wrong. First check one selection, then repeat that same operation across the named groups.

Room question: ask pairs for a repeated research task, the item that changes each time and a check they could apply to every result.

Make One Iteration Visible

Repeat the operation, preserving the identity and checks of every result.

Automation repeats mistakes as readily as correct work. Check one small example before increasing the number of iterations.

A for Loop

The loop takes each value in sequence in order. The body inside {} is executed for that value before the next iteration begins. The empty body above is a syntax scaffold: it performs no useful action yet. Use a descriptive loop-variable name in a working script.

Work Through a Small Example

1 * 1 = 1 
2 * 2 = 4 
3 * 3 = 9 
4 * 4 = 16 
5 * 5 = 25 
6 * 6 = 36 
7 * 7 = 49 
8 * 8 = 64 
9 * 9 = 81 
10 * 10 = 100 

Predict the first and last line before running it. The code prints each value and its square; it does not return a saved table of results.

A Value Is Not Its Position

value: 2 square: 4 
value: 4 square: 16 
value: 6 square: 36 

position: 1 value: 2 
position: 2 value: 4 
position: 3 value: 6 

The first loop visits 2, 4, 6; the second visits positions 1, 2, 3 and retrieves their values. Use positions when you need to place each result in a matching output position.

seq_along() also gives an empty sequence for an empty vector. In contrast, 1:length(numeric(0)) produces 1:0, containing two positions. Ask R to generate the valid positions rather than assuming there is always a first element. See the R sequence documentation.

Task 1: Trace Before Running (5 min)

Goal: distinguish an iteration’s value, position and output. Input: the square-printing loop and chosen_values above. Work in pairs.

• Do the Task
• Debrief

1. 1 min: predict the first, second and final printed lines.
2. 2 min: run the examples for c(2, 4, 6) and write one line showing position, input and square.
3. 1 min: compare seq_along(numeric(0)) with 1:length(numeric(0)); predict how many times each would run a loop.
4. 1 min: the instructor asks whether a loop over chosen_values visits positions or values. Vote by show of hands; two pairs explain their evidence.

Keep: the annotated loop, three-row trace and empty-input note. If access fails: trace the iterations on paper using the displayed values.

For 1:10, the first square is 1 and the final square is 100. For chosen_values, positions 1, 2 and 3 contain values 2, 4 and 6. The calculation stays the same while the input changes. seq_along() produces no iterations for an empty input; 1:0 would incorrectly attempt two. Predicting a small result gives a check more useful than simply seeing many lines of output.

Retain Results and Choose a Clear Approach

Save a Result for Every Input

Printing answers is useful during diagnosis, but later calculations need retained values. Allocate an output vector of the right length and place each result at the matching position:

  input square
1     2      4
2     4     16
3     6     36

Initialising the container once keeps the output’s length and order clear. Assigning only answer <- nums[index]^2 on each iteration would replace the previous answer, leaving just the last square. Reinitialising squares inside the loop would discard earlier results too.

Vectorisation

Many R operations act on an entire vector. The original example can also be written:

1 * 1 = 1
2 * 2 = 4
3 * 3 = 9
4 * 4 = 16
5 * 5 = 25
6 * 6 = 36
7 * 7 = 49
8 * 8 = 64
9 * 9 = 81
10 * 10 = 100

This calculates all squares in one expression. In paste(), sep controls separators between arguments, while collapse combines the resulting elements into one string. Read ?paste and ?cat when the output differs from your prediction.

Reuse a Function over a Vector

 [1]   1   4   9  16  25  36  49  64  81 100

A function makes the operation reusable; vectorisation can make this numerical step clear without a loop. A loop remains useful when each iteration performs a separate action, such as writing a different file.

Need	Useful approach	Check
Calculate a square for every numerical value	nums^2 or square_values(nums)	Values, length and ordering match expectations
Retain a result alongside a separate record for each input	A loop with an allocated output container	Each result occupies its intended position
Save a separate file for each group	A loop with one named destination per iteration	Selected group, path and read-back content

Choose by the task, its clarity and its checks rather than assuming one technique is always better.

Task 2: Compare Equivalent Calculations (5 min)

Goal: retain complete numerical results and justify an implementation. Input: the allocated squares loop, vectorised expression and square_values() above. Swap operator and checker.

• Do the Task
• Debrief

1. 1 min: identify which examples print text and which retain numerical values.
2. 2 min: compare the collected loop results with square_values(c(2, 4, 6)); add a fourth input and rerun the whole collection block.
3. 1 min: inspect paste() help and decide which approach suits a later numerical calculation and which suits separate file exports.
4. 1 min: the instructor asks which code would retain only one square if its assignment used no index. Two pairs identify the cause and explain their preferred approach.

Keep: the complete square_record and a sentence explaining your choice. If access fails: make the table on paper and mark where each output would be stored.

The first three squares are 4, 16 and 36. The allocated loop and vectorised function agree because they implement the same calculation. An assignment without the output index replaces the previous result. Retaining numerical values supports further calculation; printing is presentation. Separate file exports have a distinct filename and side effect on each iteration, which makes a loop appropriate.

Conditions and Loop Diagnosis

Record a Missing Input Explicitly

This fictional teaching vector includes a missing input. Preserve its position and record why no square was calculated:

  input square        status
1     2      4    calculated
2    NA     NA missing input
3     6     36    calculated

The missing result is retained as NA; it is not replaced by zero or removed. This example records missingness rather than deciding how a real study should handle it.

A Condition Needs One Definite Answer

An if statement chooses one branch from a single TRUE or FALSE. In this loop, is.na(current_value) asks about one element. if (values_to_check > 0) would supply several answers, and if (NA) supplies no definite answer. Both require investigation. See the R control-flow documentation.

next moves to the next iteration; break ends the loop. Either can leave incomplete results. If you use one, record what was skipped or why the loop ended. For this example, if/else makes a result and status visible for every input.

Diagnose the First Wrong Iteration

Symptom	Inspect first	Useful check
Only the final result remains	Where the output is initialised and assigned	Output length and input-to-output positions
A condition fails	The current value, length() and is.na()	One non-missing logical value for if
A selection is empty	The exact label and available labels	Count and identity before any export
A later iteration fails	The same body run on that single input	Reproduce outside the loop with its index/name

Keep the error message and inspect str(current_value) rather than suppressing the failure. If a file-writing loop stops, some earlier files may already exist: a partially completed run must not be described as complete.

Room prediction: show the missing input before running the code. Ask whether its result should be zero, disappear or remain missing with an explanation; take the evidence behind two answers.

Task 3: Diagnose and Record a Missing Result (5 min)

Goal: distinguish a missing input from a failed or incomplete loop. Input: the fictional values_to_check loop and diagnostic table. Work in pairs.

• Do the Task
• Debrief

1. 1 min: predict the three rows of check_record, including the missing result’s status.
2. 2 min: run the block and compare the result; inspect the missing element with is.na() and confirm that all three input positions remain.
3. 1 min: choose a diagnostic for a loop that keeps only the last answer and explain why setting missing inputs to zero would change the data.
4. 1 min: the instructor asks, “Is every missing output a loop error?” Two pairs use the status record to explain their answer.

Keep: check_record and one symptom–check pair in your script comments. If access fails: annotate the three expected output rows and the missingness decision on paper.

The squares are 4, missing and 36, with statuses calculated, missing input and calculated. The missing result reflects its input; an omitted position or repeatedly overwritten output reflects an implementation problem. Zero would invent a measurement. A status record makes these cases distinguishable without hiding the issue.

Break (10 min)

Save your script. After the break, transfer the same input–operation–check–output pattern to the real Gapminder teaching data.

Repeat Operations over Groups and Lists

Inspect One Group First

The %>% pipe passes the object on its left into the next function. In the selection below, gapminder %>% filter(continent == first_continent) means filter(gapminder, continent == first_continent): the table becomes the first argument. Loading dplyr makes this pipe available for these examples.

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

[1] "Asia"     "Europe"   "Africa"   "Americas" "Oceania" 

# A tibble: 6 × 6
  country     continent  year lifeExp      pop gdpPercap
  <fct>       <fct>     <int>   <dbl>    <int>     <dbl>
1 Afghanistan Asia       1952    28.8  8425333      779.
2 Afghanistan Asia       1957    30.3  9240934      821.
3 Afghanistan Asia       1962    32.0 10267083      853.
4 Afghanistan Asia       1967    34.0 11537966      836.
5 Afghanistan Asia       1972    36.1 13079460      740.
6 Afghanistan Asia       1977    38.4 14880372      786.

[1] 396

Gapminder records historical country–year observations; the package documentation describes its variables and coverage. Keep country and year so that records remain identifiable. Verify the selected continent and count before writing a file. The filter() expression retains rows whose continent matches the current name; it does not summarise their measurements.

Store Tables in a Named List

A list can retain tables with different numbers of rows. Allocate one element per continent and label it with the corresponding group:

  continent n_rows
1      Asia    396
2    Europe    360
3    Africa    624
4  Americas    300
5   Oceania     24

# A tibble: 6 × 6
  country continent  year lifeExp      pop gdpPercap
  <fct>   <fct>     <int>   <dbl>    <int>     <dbl>
1 Algeria Africa     1952    43.1  9279525     2449.
2 Algeria Africa     1957    45.7 10270856     3014.
3 Algeria Africa     1962    48.3 11000948     2551.
4 Algeria Africa     1967    51.4 12760499     3247.
5 Algeria Africa     1972    54.5 14760787     4183.
6 Algeria Africa     1977    58.0 17152804     4910.

[[index]] puts a whole table inside one list element. tables_by_continent["Africa"] returns a one-element list; tables_by_continent[["Africa"]] returns the table inside it. Use the latter when a function expects a table.

The row-count check confirms that these partitions account for the source’s row total. It does not alone prove correct allocation: duplicated rows and omissions could cancel out, so also inspect group identities and country–year records.

One Operation across a List

For a straightforward operation on every element, base R’s lapply() is another option:

$Asia
[1] 396

$Europe
[1] 360

$Africa
[1] 624

$Americas
[1] 300

$Oceania
[1] 24

This returns a named list of row counts. It does not create files or combine the original observations. The explicit loop is retained here because it lets us inspect selections, counts and identities together. Keep a clear implementation before exploring other iteration tools.

Task 4: Check One Group and Its Identity (5 min)

Goal: inspect a grouped operation without losing observation identity. Input: Gapminder, tables_by_continent and group_record. Swap operator and checker.

• Do the Task
• Debrief

1. 1 min: choose a continent from continents and predict what one row represents.
2. 2 min: run the list-building block; retrieve the chosen table with [[ ]] and inspect its row count and continent values.
3. 1 min: select its first country–year pair and find that same pair in gapminder; compare lifeExp without changing or rounding it.
4. 1 min: the instructor asks whether equal total row counts alone prove correct group contents. Vote, then two pairs name the additional check they used.

Keep: the grouped tables, row-count record and annotated source comparison. If access fails: inspect the instructor’s chosen group and locate the country–year record in the displayed source.

Each row remains a historical country–year observation. A continent table should contain its named group and preserve the selected record’s values. Total counts are useful evidence of coverage, while group identities and source comparisons provide different checks. Counts do not turn these tables into population-weighted or individual-level summaries.

Plan and Write Separate Outputs

Write to a Separate Output Folder

These files are derived teaching outputs. Keep them outside data. By default, write_csv() replaces an existing destination file; see its official documentation. Create a new run folder and refuse to reuse an existing destination before writing.

The label below uses the run’s local date and time. When rerunning, begin at the label assignment so a new folder is chosen. If two runs select the same label, the check stops: choose another descriptive label rather than deleting existing work.

[1] "output/continents/20261012-135417"

Plan the Files before Writing

  continent                                                 filename n_rows
1      Asia     output/continents/20261012-135417/gapminder_Asia.csv      0
2    Europe   output/continents/20261012-135417/gapminder_Europe.csv      0
3    Africa   output/continents/20261012-135417/gapminder_Africa.csv      0
4  Americas output/continents/20261012-135417/gapminder_Americas.csv      0
5   Oceania  output/continents/20261012-135417/gapminder_Oceania.csv      0
  read_back_checked
1             FALSE
2             FALSE
3             FALSE
4             FALSE
5             FALSE

The filenames use the known continent labels from this teaching dataset. For a project receiving filenames from outside the workflow, validate those names before joining paths. file.path() constructs paths for the operating system; paste0() joins the filename text without a separator.

Repeat the Checked Export

output/continents/20261012-135417/gapminder_Asia.csv
output/continents/20261012-135417/gapminder_Europe.csv
output/continents/20261012-135417/gapminder_Africa.csv
output/continents/20261012-135417/gapminder_Americas.csv
output/continents/20261012-135417/gapminder_Oceania.csv

  continent                                                 filename n_rows
1      Asia     output/continents/20261012-135417/gapminder_Asia.csv    396
2    Europe   output/continents/20261012-135417/gapminder_Europe.csv    360
3    Africa   output/continents/20261012-135417/gapminder_Africa.csv    624
4  Americas output/continents/20261012-135417/gapminder_Americas.csv    300
5   Oceania  output/continents/20261012-135417/gapminder_Oceania.csv     24
  read_back_checked
1             FALSE
2             FALSE
3             FALSE
4             FALSE
5             FALSE

The loop records what it actually wrote and how many rows were selected. read_back_checked remains FALSE until the next block verifies the files. Distinguish a planned path, a file on disk and a checked result.

If writing fails midway, retain the error and inspect the incomplete run folder. Fix the cause and start a new run; do not present the partial folder as a complete export or append blindly to its files.

Task 5: Review the Destination before Exporting (5 min)

Goal: export separate tables with an explicit destination and no planned overwrite. Input: the new run folder, output_record and checked Gapminder groups. Work in pairs.

• Do the Task
• Debrief

1. 1 min: inspect getwd() and the proposed destination; confirm that the input dataset and source files are separate.
2. 2 min: run the folder, file-plan and export blocks once; find the resulting files in RStudio’s Files tab.
3. 1 min: compare filenames and counts with group_record; identify which block refuses an already existing destination.
4. 1 min: the instructor asks whether five existing files establish a checked export. Two pairs state what is still needed before handover.

Keep: the files, script and output record. If access fails: annotate the instructor’s file plan with the group, destination and overwrite check for one iteration.

A fresh run folder retains earlier practice outputs and makes this run identifiable. File existence confirms that something was written; names and counts provide further checks, but content still needs a read-back comparison. Restarting only the export loop encounters its existing-file guard; start the complete workflow at the new label instead.

Read Back, Rerun and Hand Over

Read Back and Inspect

# A tibble: 6 × 6
  country     continent  year lifeExp      pop gdpPercap
  <chr>       <chr>     <int>   <dbl>    <dbl>     <dbl>
1 Afghanistan Asia       1952    28.8  8425333      779.
2 Afghanistan Asia       1957    30.3  9240934      821.
3 Afghanistan Asia       1962    32.0 10267083      853.
4 Afghanistan Asia       1967    34.0 11537966      836.
5 Afghanistan Asia       1972    36.1 13079460      740.
6 Afghanistan Asia       1977    38.4 14880372      786.

CSV stores values rather than all R object metadata. Factor levels, for example, are not preserved as factor metadata. The explicit import specification states the expected types; inspecting problems() prevents parse failures being mistaken for valid missing values. The source’s factors are converted to text only for the identity comparison; the source table remains unchanged.

The checks above compare row counts, group identity, country–year ordering and life expectancy. Other numerical columns still require checks, which the repeated verification below supplies.

Verify Each File and Save the Record

  continent                                                 filename n_rows
1      Asia     output/continents/20261012-135417/gapminder_Asia.csv    396
2    Europe   output/continents/20261012-135417/gapminder_Europe.csv    360
3    Africa   output/continents/20261012-135417/gapminder_Africa.csv    624
4  Americas output/continents/20261012-135417/gapminder_Americas.csv    300
5   Oceania  output/continents/20261012-135417/gapminder_Oceania.csv     24
  read_back_checked
1              TRUE
2              TRUE
3              TRUE
4              TRUE
5              TRUE

The small inner loop repeats numerical checks for three named columns. It does not alter the measurements. Floating-point comparisons use all.equal() with its default tolerance, rather than rounding or changing data to force a pass. Only after a file passes every check is its status set to TRUE.

What Makes This a Repeatable Workflow?

Keep together	Why a collaborator needs it
Saved script, starting with package loading	Recreates the objects rather than relying on console history
Input identity and grouping rule	Explains why each observation is in its file
Run folder and checked output_record.csv	Locates the products of this specific run
sessionInfo() in working notes	Records R and package versions for troubleshooting
Interpretation and known limitations	Distinguishes preserved data from a scientific conclusion

Save the script before restarting R. With the project open, run it from the beginning, including the new run-label assignment. A new folder should be created and equivalent input records should produce equivalent tables. The label and file paths will differ between runs; the source data and calculations should not.

Task 6: Check and Hand Over the Run (5 min)

Goal: verify exported content and make the run discoverable. Input: this run’s exports, output_record, source tables and saved script. Swap operator and checker.

• Do the Task
• Debrief

1. 1 min: read back the first file and identify the checks for group, row identity and numerical values.
2. 2 min: run the repeated verification; save the output record and script, then record sessionInfo() in working notes.
3. 1 min: the partner locates one exported country–year observation in the source and explains where to begin a new run. Check the script defines every required object.
4. 1 min: the instructor asks which check would detect a wrong-group export with the correct row count. Two pairs report their evidence and one limitation of the workflow.

Keep: the checked files, saved script, output_record.csv and version notes. If access fails: annotate the displayed record with the identity and value comparisons you would require before setting a status to TRUE.

A useful output record states what was written, where, with how many rows and whether read-back checks passed. Country–year and continent comparisons can reveal incorrect selection even when counts agree. Numerical comparisons check exported values within their stated tolerance. These checks establish properties of the export, not that the dataset or a later scientific interpretation is appropriate for every research question.

Further Information

📚 Key Points

• Trace and check one iteration before repeating a task.
• Retain outputs by position or name; printing is different from storing results.
• Use vectorised operations for suitable calculations and loops for separate repeated actions.
• Record missing inputs and incomplete runs rather than hiding or silently changing them.
• Preserve input data and identifiers; plan destinations and read back derived outputs.

🔦 Hints

• Use seq_along() when iterating over positions in a vector, including an empty one.
• Inspect one problematic input outside the loop; keep its name, position and error message.
• Check getwd() and file paths when an expected output is missing.
• Rerun from the new-label assignment; an existing destination should trigger investigation.
• Save the script rather than relying on objects from an earlier console session.

Module Summary

You can trace repeated calculations, retain results, explain missing inputs and export checked historical tables systematically. Your script and output record make each run easier for a collaborator to inspect. Next, use those foundations for lesson 5: Cleaning, Wrangling and Joining Data.

Next-week action: choose one small repeated task in your own project. Check one item, record input identities and expected outputs, then repeat the operation with an explicit failure and file-handling plan.

Additional Learning

• Learner Additional Material: practise a complete Gapminder comparison.
• Reference: syntax reminders.
• An Introduction to R: vector operations, lists and repeated execution.
• R control-flow documentation: conditions, loop values, next and break.
• readr file-writing documentation: CSV output and replacement/append behaviour.

Course Overview · Previous: Functions · Next: Data Analysis

Feedback and Register