To cite package markovchain in publications use:

Giorgio Alfredo Spedicato. markovchain: discrete time Markov chains made easy

Corresponding BibTeX entry:

  @Manual{,
    title = {markovchain: discrete time Markov chains made easy},
    author = {Giorgio Alfredo Spedicato},
    month = {06},
    year = {2014},
    note = {R package version 0.0.9.5},
  }