Skip to contents

Your first simulation

To perform your first simulation you will need:

  1. A data.frame specifiying the experiment design, and
  2. A list with the parameters for the model you’ll be using.

The design data.frame

For this example we will use a blocking design.

library(calm)

my_blocking <- data.frame(
  Group = c("Exp", "Control"),
  Phase1 = c("10A(US)", "10C(US)"),
  R1 = c(FALSE, FALSE),
  Phase2 = c("10AB(US)", "10AB(US)"),
  R2 = c(FALSE, FALSE),
  Test = c("1#A/1#B", "1#A/1#B"),
  R3 = c(FALSE, FALSE)
)
# parsing the design and showing the original and what was detected
parsed <- parse_design(my_blocking)
parsed
#> CalmDesign built from data.frame:
#>     Group  Phase1    R1   Phase2    R2    Test    R3
#> 1     Exp 10A(US) FALSE 10AB(US) FALSE 1#A/1#B FALSE
#> 2 Control 10C(US) FALSE 10AB(US) FALSE 1#A/1#B FALSE
#> ----------------
#> Trials detected:
#>     group  phase trial_names trial_repeats is_test stimuli
#> 1     Exp Phase1       A(US)            10   FALSE    A;US
#> 2     Exp Phase2      AB(US)            10   FALSE  A;B;US
#> 3     Exp   Test          #A             1    TRUE       A
#> 4     Exp   Test          #B             1    TRUE       B
#> 5 Control Phase1       C(US)            10   FALSE    C;US
#> 6 Control Phase2      AB(US)            10   FALSE  A;B;US
#> 7 Control   Test          #A             1    TRUE       A
#> 8 Control   Test          #B             1    TRUE       B

A few rules about the design data.frame:

  1. Each row represents a group.
  2. The first column contains the group labels.
  3. The remaining columns are organized in pairs (trials in a phase, and whether to randomize them)

The trials in each phase column are specified using a very rigid notation. A few observations about it:

  1. Trials are preceded by a number. That number represents the number of times that trial will be given in each phase. “10A(US)” means that the “A(US)” trial will be given 10 times.
  2. The presence and absence of the unconditioned stimulus are not denoted with the traditional “+” and “-” symbols. Instead, here we use parenthesis to denote “complex” stimuli. These can be thought of as an element with a complex name (i.e., with more than one character). As such, “(US)” specifies a single element to represent the US.
  3. In the same vein, multiple characters with no parentheses denote individual elements. For example, “AB” implies the presence of two stimuli, A and B.
  4. The “/” character is used as a trial separator (it does not imply randomization by itself). Thus, “1A/1B” specifies that a single “A” trial and a single “B” trial will be given during that phase. Recall that randomization of trials within a phase is specified by the column after it (above, R1, R2, and R3).
  5. The “#” character is used to denote probe trials. In contrast to real life, probe trials here entail no update of the model’s associations. As such, probe trials can be used to track the development of key associations, with no repercussion to what the model learns on normal training trials.

If you want to check whether your phase string will work with the simulator, you can use phase_parser. The function returns a list with a lot of information, so let’s print only some of the fields.

# not specifying a number of AB trials. Error!
phase_parser("AB/10AC")
#> Error in if (is.na(treps)) 1 else treps: argument is of length zero
# putting the probe symbol out of order. Error!
phase_parser("#10A")
#> Error in if (is.na(treps)) 1 else treps: argument is of length zero
# considering a configural cue for elements AB
trial <- phase_parser("10AB(AB)(US)")
# different USs
trial <- phase_parser("10A(US1)/10B(US2)")

The parameters list

Now we need to pick a model and its parameters.

To get the models currently supported in calm, you can call supported_models().

supported_models()
#> [1] "HDI2020" "HD2022"  "RW1972"  "MAC1975" "PKH1982" "SM2007"  "RAND"   
#> [8] "ANCCR"

After choosing a model, you can get default parameters for your design with get_parameters.

my_pars <- get_parameters(my_blocking, model = "RW1972")
# Increasing the beta parameter for US presentations
my_pars$betas_on["US"] <- .6
my_pars
#> $alphas
#>   A   B   C  US 
#> 0.4 0.4 0.4 0.4 
#> 
#> $betas_on
#>   A   B   C  US 
#> 0.4 0.4 0.4 0.6 
#> 
#> $betas_off
#>   A   B   C  US 
#> 0.4 0.4 0.4 0.4 
#> 
#> $lambdas
#>  A  B  C US 
#>  1  1  1  1

Simulating

With all of the above, we can run our simulation using the run_experiment function. This function also takes extra arguments to manipulate the number of iterations to run the experiment for, and whether to organize trials in miniblocks (see help(make_experiment) for additional details). Below, we run the experiment for 5 iterations.

my_experiment <- run_experiment(
  my_blocking, # note we do not need to pass the parsed design
  model = "RW1972",
  parameters = my_pars,
  iterations = 5
)
# returns an CalmExperiment object
class(my_experiment)
#> [1] "CalmExperiment"
#> attr(,"package")
#> [1] "calm"
# CalmExperiment is an S4 class, so it has slots
slotNames(my_experiment)
#> [1] "design"      "model"       "groups"      "parameters"  "experiences"
#> [6] "results"     ".model"      ".group"      ".iter"
# the experience given to group Exp on the first iteration
my_experiment@experiences[[1]]
#>     model group  phase tp     tn is_test block_size trial
#> 1  RW1972   Exp Phase1  1  A(US)   FALSE          1     1
#> 2  RW1972   Exp Phase1  1  A(US)   FALSE          1     2
#> 3  RW1972   Exp Phase1  1  A(US)   FALSE          1     3
#> 4  RW1972   Exp Phase1  1  A(US)   FALSE          1     4
#> 5  RW1972   Exp Phase1  1  A(US)   FALSE          1     5
#> 6  RW1972   Exp Phase1  1  A(US)   FALSE          1     6
#> 7  RW1972   Exp Phase1  1  A(US)   FALSE          1     7
#> 8  RW1972   Exp Phase1  1  A(US)   FALSE          1     8
#> 9  RW1972   Exp Phase1  1  A(US)   FALSE          1     9
#> 10 RW1972   Exp Phase1  1  A(US)   FALSE          1    10
#> 11 RW1972   Exp Phase2  2 AB(US)   FALSE          1    11
#> 12 RW1972   Exp Phase2  2 AB(US)   FALSE          1    12
#> 13 RW1972   Exp Phase2  2 AB(US)   FALSE          1    13
#> 14 RW1972   Exp Phase2  2 AB(US)   FALSE          1    14
#> 15 RW1972   Exp Phase2  2 AB(US)   FALSE          1    15
#> 16 RW1972   Exp Phase2  2 AB(US)   FALSE          1    16
#> 17 RW1972   Exp Phase2  2 AB(US)   FALSE          1    17
#> 18 RW1972   Exp Phase2  2 AB(US)   FALSE          1    18
#> 19 RW1972   Exp Phase2  2 AB(US)   FALSE          1    19
#> 20 RW1972   Exp Phase2  2 AB(US)   FALSE          1    20
#> 21 RW1972   Exp   Test  3     #A    TRUE          2    21
#> 22 RW1972   Exp   Test  4     #B    TRUE          2    22
# the number of times we ran the model (groups x iterations)
length(experiences(my_experiment))
#> [1] 10
# an experiment has results with different levels of aggregation
class(my_experiment@results)
#> [1] "CalmExperimentResult"
#> attr(,"package")
#> [1] "calm"
slotNames(my_experiment@results)
#> [1] "aggregated_results" "parsed_results"     "raw_results"
# shorthand method to access aggregated_results
results(my_experiment)
#> $rs
#>        group  phase trial_type trial     s1     s2 block_size value  model
#>       <char> <char>     <char> <int> <char> <char>      <num> <num> <char>
#>   1:     Exp Phase1      A(US)     1      A      A          1     0 RW1972
#>   2:     Exp Phase1      A(US)     1      A      B          1     0 RW1972
#>   3:     Exp Phase1      A(US)     1      A      C          1     0 RW1972
#>   4:     Exp Phase1      A(US)     1      A     US          1     0 RW1972
#>   5:     Exp Phase1      A(US)     1      B      A          1     0 RW1972
#>  ---                                                                      
#> 700: Control   Test         #B    22      C     US          2     0 RW1972
#> 701: Control   Test         #B    22     US      A          2     0 RW1972
#> 702: Control   Test         #B    22     US      B          2     0 RW1972
#> 703: Control   Test         #B    22     US      C          2     0 RW1972
#> 704: Control   Test         #B    22     US     US          2     0 RW1972
#> 
#> $vs
#>        group  phase trial_type trial     s1     s2 block_size     value  model
#>       <char> <char>     <char> <int> <char> <char>      <num>     <num> <char>
#>   1:     Exp Phase1      A(US)     1      A      A          1 0.0000000 RW1972
#>   2:     Exp Phase1      A(US)     1      A      B          1 0.0000000 RW1972
#>   3:     Exp Phase1      A(US)     1      A      C          1 0.0000000 RW1972
#>   4:     Exp Phase1      A(US)     1      A     US          1 0.0000000 RW1972
#>   5:     Exp Phase1      A(US)     1      B      A          1 0.0000000 RW1972
#>  ---                                                                          
#> 700: Control   Test         #B    22      C     US          2 0.9939534 RW1972
#> 701: Control   Test         #B    22     US      A          2 0.4999999 RW1972
#> 702: Control   Test         #B    22     US      B          2 0.4999999 RW1972
#> 703: Control   Test         #B    22     US      C          2 0.6626356 RW1972
#> 704: Control   Test         #B    22     US     US          2 0.0000000 RW1972

If you are an advanced R user you will be able to dig into the data straight away.

However, the package also includes some methods to get a quick look at the results.

Plotting

Let’s use plot method to create some plots. Each model supports different types of plots according to the results they can produce (e.g., associations, responses, saliences, etc.)

# get all the plots for the experiment
plots <- plot(my_experiment)
names(plots)
#> [1] "Exp - Response Strength (RW1972)"       
#> [2] "Control - Response Strength (RW1972)"   
#> [3] "Exp - Association Strength (RW1972)"    
#> [4] "Control - Association Strength (RW1972)"
# or get a specific type of plot
specific_plot <- plot(my_experiment, type = "vs")
names(specific_plot)
#> [1] "Exp - Association Strength (RW1972)"    
#> [2] "Control - Association Strength (RW1972)"
# show which plots are supported by the model we are using
supported_plots("RW1972")
#> [1] "rs" "vs"

In this case, the RW model supports both associations and responses.

Stimulus associations

The columns below are the phases of the design and the rows denote the source of the association. The colors within each panel determine the target of the association.

plot(my_experiment, type = "vs")
#> $`Exp - Association Strength (RW1972)`

#> 
#> $`Control - Association Strength (RW1972)`

Responding

Fairly similar to the above, but this time responding is a function of the stimuli presented on a trial.

plot(my_experiment, type = "rs")
#> $`Exp - Response Strength (RW1972)`

#> 
#> $`Control - Response Strength (RW1972)`

Graphing

You can also take a look at the state of the model’s associations at any point during training, using the graph method in your experiment.

my_graph_opts <- get_graph_opts("small")
graph(my_experiment, t = 20, graph_opts = my_graph_opts)
#> $RW1972
#> $RW1972$`Exp - Associations (RW1972)`

#> 
#> $RW1972$`Control - Associations (RW1972)`

Final thoughts

The calm package was designed to simulate quickly: write your design and get a glance at model predictions.

Yet, the package also has some features for advanced R users, so make sure to check the other vignettes when you are ready.