Gym Class Reference

Class for a learning interface. More...

#include <Gym.hh>

Classes
struct	GymInfo
	Class for encapsulation of return values of Gym. More...

Public Member Functions
	Gym (Model model, Parity dpw, GymOptions options, std::vector< Parity > exdpws={})
	Constructor from model.
void	resetStats (void)
	Resets statistics.
void	printStats (void) const
	Prints statistics about a run.
void	printDotLearn (Qtype const &Q, std::string filename=std::string("-")) const
	Prints the dot representation of learned pruned product graph. More...
void	printPrismLearn (Qtype const &Q, std::string filename=std::string("-")) const
	Prints the PRISM representation of learned pruned product graph. More...
std::map< double, std::map< unsigned int, double > >	getProbabilityOfSat (Qtype const &Q, bool statsOn) const
	Returns the probability of satisfaction in the learned pruned product graph for each tolerance and property specified via GymOptions at construction. More...
Model	QtoModel (Qtype const &Q, double tolerance, bool p1strategic, bool statsOn, double priEpsilon=0.001, unsigned int objIndex=0) const
	Constructs Markov chain from Q-table. More...
std::vector< GymTransitionTuple >	getParallelUpdates (GymObservation S, GymAction A, GymObservation sPrime)
	Returns all transition tuples that were possible in the product given that the MDP transition is fixed.
std::vector< GymAction >	getAllActions (GymObservation obs)
	Returns all available actions from a particular state.
std::vector< GymAction >	getActions (void)
	Returns available actions from current state, excluding special actions.
GymInfo	reset (void)
	Resets environment.
GymInfo	step (GymAction)
	Takes step in environment according to action.
const Model &	getModel (void) const
	Returns a reference to model.
Player	getPlayer (GymObservation const &observation) const
	Returns the player controlling state GymObservation.
std::string	toString (GymObservation const &observation) const
	Returns a string representing GymObservation.
std::string	toString (GymAction const &action) const
	Returns a string representing GymAction.
std::string	toString (Qtype const &Q) const
	Returns a string representing Qtype.
void	saveQ (Qtype const &Q, std::string saveType, std::string filename) const
	Saves Q-table.
void	saveQ (Qtype const &Q1, Qtype const &Q2, std::string saveType, std::string filename) const
	Saves Q-tables.
void	saveQ (Qtype const &Q, double Rbar, std::string saveType, std::string filename) const
	Saves Q-table and .
void	loadQ (Qtype &Q, std::string saveType, std::string filename) const
	Loads Q-table.
void	loadQ (Qtype &Q1, Qtype &Q2, std::string saveType, std::string filename) const
	Loads Q-tables.
void	loadQ (Qtype &Q, double &Rbar, std::string saveType, std::string filename) const
	Loads Q-table and .
void	saveStrat (Qtype &Q, std::string filename) const
	Saves mixed strategy from Q-table to CSV file for all visited states.
void	saveStratBDP (Qtype &Q, std::string filename) const
	Saves mixed strategy from Q-table to file for all visited states.

Detailed Description

Class for a learning interface.

Member Function Documentation

getProbabilityOfSat()

map< double, map< unsigned int, double > > Gym::getProbabilityOfSat	(	Qtype const &	Q,
		bool	statsOn
	)		const

Returns the probability of satisfaction in the learned pruned product graph for each tolerance and property specified via GymOptions at construction.

Parameters

Q	Q-table the strategy is extracted from.
statsOn	Indicates verbose printing.

Returns

Map from each tolerance to associated probability of satisfaction.

printDotLearn()

void Gym::printDotLearn	(	Qtype const &	Q,
		std::string	filename = std::string("-")
	)		const

Prints the dot representation of learned pruned product graph.

The first tolerance specified via GymOptions at construction is used.

Parameters

Q	Q-table the strategy is extracted from.

printPrismLearn()

void Gym::printPrismLearn	(	Qtype const &	Q,
		std::string	filename = std::string("-")
	)		const

Prints the PRISM representation of learned pruned product graph.

The first tolerance specified via GymOptions at construction is used.

Parameters

Q	Q-table the strategy is extracted from.

QtoModel()

Model Gym::QtoModel	(	Qtype const &	Q,
		double	tolerance,
		bool	p1strategic,
		bool	statsOn,
		double	priEpsilon = 0.001,
		unsigned int	objIndex = 0
	)		const

Constructs Markov chain from Q-table.

Parameters

Q	Q-table the strategy is extracted from.
tolerance	Relative fraction from the maximum where values in Q-table are considered indistinguishable. A uniform random strategy is produced if multiple actions are indistinguishable in value.
p1strategic	If true, then player 1's strategy is not fixed based on the Q-table. This results in an MDP where only player 1 has choices.
statsOn	Indicates verbose printing.
priEpsilon	Probability used to advance priority tracker.
objIndex	The index of the objective which will determine the edge priorities.

---

The documentation for this class was generated from the following files:

• Gym/Gym.hh
• Gym/Gym.cc