Touch system

mimoTouch.touch

This module defines the touch system interface and provides two implementations.

mimoTouch.sensorpoints

This module contains functions for distributing points over geometric primitives.

mimoTouch.sensormeshes

This module contains functions for distributing points over geometric primitives.

mimoTouch.touch

mimoTouch.touch.Touch

Abstract base class for the touch system.

mimoTouch.touch.DiscreteTouch

A simple touch class using MuJoCo geoms as the basic sensing component.

mimoTouch.touch.TrimeshTouch

A touch class with sensor meshes using MuJoCo bodies as the basic sensing component.

This module defines the touch system interface and provides two implementations.

The interface is defined as an abstract class in Touch. A simple implementation with a cloud of sensor points is in DiscreteTouch. A second implementation using trimesh objects is in TrimeshTouch. This second implementation allows for consideration of sensor normals as well as surface distance, avoiding the issue of a contact penetrating through to sensors on the opposite side of the sensing body.

Both of the implementations also have functions for visualizing the touch sensations.

class mimoTouch.touch.Touch(env, touch_params)

Bases: object

Abstract base class for the touch system.

This class defines the functions that all implementing classes must provide. get_touch_obs() should perform the whole sensory pipeline as defined in the configuration and return the output as a single array. Additionally, the output for each body part should be stored in sensor_outputs. The exact definition of ‘body part’ is left to the implementing class.

The constructor takes two arguments, env and touch_params: env should be an openAI gym environment using MuJoCo, while touch_params is a configuration dictionary. The exact form will depend on the specific implementation, but it must contain these three entries:

  • ‘scales’, which lists the distance between sensor points for each body part.

  • ‘touch_function’, which defines the output type and must be in VALID_TOUCH_TYPES.

  • ‘response_function’, which defines how the contact forces are distributed to the sensors. Must be one of VALID_RESPONSE_FUNCTIONS.

The sensor scales determines the density of the sensor points, while ‘touch_function’ and ‘response_function’ determine the type of contact output and how it is distributed. ‘touch_function’ and ‘response_function’ refer to implementation methods by name and must be listed in VALID_TOUCH_TYPES. and VALID_RESPONSE_FUNCTIONS respectively. Different touch functions should be used to support different types of output, such as normal force, frictional forces or contact slip. The purpose of the response function is to loosely simulate surface behaviour. How exactly these functions work and interact is left to the implementing class. Note that the bodies listed in ‘scales’ must actually exist in the scene to avoid errors!

env

The environment to which this module will be attached.

Type

MujocoEnv

sensor_scales

A dictionary listing the sensor distances for each body part. Populated from touch_params.

Type

Dict[int, float]

touch_type

The name of the member method that determines output type. Populated from touch_params.

Type

str

touch_function

A reference to the actual member method determined by touch_type.

Type

Callable

touch_size

The size of the output of a single sensor for the given touch type.

Type

int

response_type

The name of the member method that determines how the output is distributed over the sensors. Populated from touch_params.

Type

str

response_function

A reference to the actual member method determined by response_type.

Type

Callable

sensor_positions

A dictionary containing the positions of the sensor points for each body part. The coordinates should be in the frame of the associated body part.

Type

Dict[int, np.ndarray]

sensor_outputs

A dictionary containing the outputs produced by the sensors for each body part. Shape will depend on the specific implementation. This should be populated by get_touch_obs(). Note that this will differ from the touch output to the environment, which is flattened.

Type

Dict[int, np.ndarray]

VALID_TOUCH_TYPES = {}

A dictionary listing valid touch output types and their sizes.

VALID_RESPONSE_FUNCTIONS = []

A list of valid surface response functions.

get_touch_obs()

Produces the current touch output.

This function should perform the whole sensory pipeline as defined in touch_params and return the output as a single array. The per-body output should additionally be stored in sensor_outputs.

Returns

A numpy array of shape (n_sensor_points, touch_size)

Return type

np.ndarray

class mimoTouch.touch.DiscreteTouch(env, touch_params)

Bases: mimoTouch.touch.Touch

A simple touch class using MuJoCo geoms as the basic sensing component.

Sensor points are simply spread evenly over individual geoms, with no care taken for cases where geoms or bodies intersect. Nearest sensors are determined by direct euclidean distance. The sensor positions in sensor_positions are directly used for the output, so altering them will also alter the output. This can be used to post-process the positions from the basic uniform distribution. Supported output types are

  • ‘normal’: The normal force as a scalar.

  • ‘force_vector’: The contact force vector (normal and frictional forces) reported in the coordinate frame of the sensing geom.

  • ‘force_vector_global’: Like ‘force_vector’, but reported in the world coordinate frame instead.

The units are newton by default, but MuJoCo does not explicitly define units. Treating the unit of distance as meters and the unit of mass as kg as we do in our models will lead to newtons as the unit of force. The output can be spread to nearby sensors in two different ways:

  • ‘nearest’: Directly add the output to the nearest sensor.

  • ‘spread_linear’: Spreads the force to nearby sensor points, such that it decreases linearly with distance to the contact point. The force drops to 0 at twice the sensor scale. The force per sensor is normalised such that the total force is conserved.

Touch functions return their output, while response functions do not return anything and instead write their adjusted forces directly into the output dictionary.

The following attributes are provided in addition to those of Touch.

m_data

A direct reference to the MuJoCo simulation data object.

Type

mujoco.MjData

m_model

A direct reference to the MuJoCo simulation model object.

Type

mujoco.MjModel

plotting_limits

A convenience dictionary listing axis limits for plotting forces or sensor points for geoms.

Type

Dict[int, float]

VALID_TOUCH_TYPES = {'force_vector': 3, 'force_vector_global': 3, 'normal': 1}

A dictionary listing valid touch output types and their sizes.

VALID_RESPONSE_FUNCTIONS = ['nearest', 'spread_linear']

A list of valid surface response functions.

add_body(body_id=None, body_name=None, scale=math.inf)

Adds sensors to all geoms belonging to the given body.

Given a body, either by ID or name, spread sensor points over all geoms and add them to the output. If both ID and name are provided the name is ignored. The distance between sensor points is determined by scale. The names are determined by the scene XMLs while the IDs are assigned during compilation.

Parameters
  • body_id (int|None) – ID of the body.

  • body_name (str|None) – Name of the body. If body_id is provided this parameter is ignored!

  • scale (float) – The distance between sensor points.

Returns

The number of sensors added to this body.

Return type

int

add_geom(geom_id=None, geom_name=None, scale=math.inf)

Adds sensors to the given geom.

Spreads sensor points over the geom identified either by ID or name and add them to the output. If both ID and name are provided the name is ignored. The distance between sensor points is determined by scale. The names are determined by the scene XMLs while the IDs are assigned during compilation.

Parameters
  • geom_id (int|None) – ID of the geom.

  • geom_name (str|None) – Name of the geom. If geom_id is provided this parameter is ignored!

  • scale (float) – The distance between sensor points.

Returns

The number of sensors added to this geom.

Return type

int

property sensing_geoms

Returns the IDs of all geoms with sensors.

Returns

The IDs for all geoms that have sensors.

Return type

List[int]

has_sensors(geom_id)

Returns True if the geom has sensors.

Parameters

geom_id (int) – The ID of the geom.

Returns

True if the geom has sensors, False otherwise.

Return type

bool

get_sensor_count(geom_id)

Returns the number of sensors for the geom.

Parameters

geom_id (int) – The ID of the geom.

Returns

The number of sensor points for this geom.

Return type

int

get_total_sensor_count()

Returns the total number of touch sensors in the model.

Returns

The total number of touch sensors in the model.

Return type

int

get_nearest_sensor(contact_id, geom_id)

Given a contact and a geom, return the sensor on the geom closest to the contact.

Contact IDs are a MuJoCo attribute, see their documentation for more detail on contacts.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom. The geom must have sensors!

Returns

The index of the closest sensor and the distance between the contact and sensor.

Return type

Tuple[int, float]

get_k_nearest_sensors(contact_id, geom_id, k)

Given a contact and a geom, find the k sensors on the geom closest to the contact.

Contact IDs are a MuJoCo attribute, see their documentation for more detail on contacts.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom. The geom must have sensors!

  • k (int) – The number of sensors to return.

Returns

The indices of the k nearest sensors, as well as the distances between their positions and the contact.

Return type

Tuple[np.ndarray, np.ndarray]

get_sensors_within_distance(contact_id, geom_id, distance)

Finds all sensors on a geom that are within a given distance to a contact.

The distance used is the direct euclidean distance. Contact IDs are a MuJoCo attribute, see their documentation for more detail on contacts.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom. The geom must have sensors!

  • distance (np.ndarray) – Sensors must be within this distance to the contact position to be included in the output.

Returns

The indices of all sensors on the geom that are within distance to the contact, as well as the distances between their positions and the contact.

Return type

Tuple[np.ndarray, np.ndarray]

get_contact_position_world(contact_id)

Get the position of a contact in the world frame.

Note that this is halfway between the touching geoms. Since geoms can intersect this point will likely be located inside both.

Parameters

contact_id (int) – The ID of the contact.

Returns

An array with the position of the contact.

Return type

np.ndarray

get_contact_position_relative(contact_id, geom_id)

Get the position of a contact in the coordinate frame of a geom.

This position is corrected for the intersection between geoms, such that the contact lies on the surface of the sensing geom.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

Returns

An array with the position of the contact.

Return type

np.ndarray

plot_sensors_geom(geom_id=None, geom_name=None)

Plots the sensor positions for a geom.

Given either an ID or the name of a geom, plot the positions of the sensors on that geom.

Parameters
  • geom_id (int|None) – The ID of the geom.

  • geom_name (str|None) – The name of the geom. This is ignored if the ID is provided!

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

plot_force_geom(geom_id=None, geom_name=None)

Plot the sensor output for a geom.

Given either an ID or the name of a geom, plots the positions and outputs of the sensors on that geom.

Parameters
  • geom_id (int|None) – The ID of the geom.

  • geom_name (str|None) – The name of the geom. This is ignored if the ID is provided!

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

plot_force_body(body_id=None, body_name=None)

Plots sensor points and output forces for all geoms in a body.

Given either an ID or the name of a body, plots the positions and outputs of the sensors for all geoms associated with that body.

Parameters
  • body_id (int|None) – The ID of the body.

  • body_name (str|None) – The name of the body. This argument is ignored if the ID is provided.

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

get_raw_force(contact_id, geom_id)

Collect the full contact force in MuJoCos own contact frame.

By convention the normal force points away from the first geom listed, so the forces are inverted if the first geom is the sensing geom.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The relevant geom in the contact. Must be one of the geoms involved in the contact!

Returns

An array with shape (3,) containing the normal force and tangential frictional forces.

Return type

np.ndarray

get_contact_normal(contact_id, geom_id)

Returns the normal vector of contact (unit vector in direction of normal) in geom coordinate frame.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

Returns

An array with shape (3,) containing the normal vector.

Return type

np.ndarray

normal(contact_id, geom_id)

Touch function. Returns the normal force as a scalar.

Given a contact and a geom, returns the normal force of the contact. The geom is required to account for the MuJoCo contact conventions.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

Returns

The normal force as a float.

Return type

float

force_vector_global(contact_id, geom_id)

Touch function. Returns the full contact force in world frame.

Given a contact returns the full contact force, i.e. the vector sum of the normal force and the two tangential friction forces, in the world coordinate frame. The geom is required to account for MuJoCo conventions and convert coordinate frames.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

Returns

An array with shape (3,) containing the contact force as a vector.

Return type

np.ndarray

force_vector(contact_id, geom_id)

Touch function. Returns full contact force in the frame of the geom.

Same as force_vector_global(), but the force is returned in the coordinate frame of the geom.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

Returns

An array with shape (3,) containing the contact force as a vector.

Return type

np.ndarray

get_contacts()

Collects all active contacts involving geoms with touch sensors.

For each active contact with a sensing geom we build a tuple (contact_id, geom_id, forces), where contact_id is the ID of the contact in the MuJoCo arrays, geom_id is the ID of the sensing geom and forces is a numpy array of the raw output force, as determined by touch_type.

Returns

A list of tuples with contact information.

Return type

List[Tuple[int, int, np.ndarray]]

get_empty_sensor_dict(size)

Returns a dictionary with empty sensor outputs.

Creates a dictionary with an array of zeros for each geom with sensors. A geom with ‘n’ sensors has an empty output array of shape (n, size). The output of this function is equivalent to the touch sensor output if there are no contacts.

Parameters

size (int) – The size of a single sensor output.

Returns

The dictionary of empty sensor outputs.

Return type

Dict[int, np.ndarray]

flatten_sensor_dict(sensor_dict)

Concatenates a touch output dictionary into a single large array in a deterministic fashion.

Output dictionaries list the arrays of sensor outputs for each geom. This function concatenates these arrays together in a reproducible fashion to avoid key order anomalies. Geoms are sorted by their ID.

Parameters

sensor_dict (Dict[int, np.ndarray]) – The output dictionary to be flattened.

Returns

The concatenated numpy array.

Return type

np.ndarray

get_touch_obs()

Produces the current touch sensor outputs.

Does the full contact getting-processing process, such that we get the forces, as determined by touch_type and response_type, for each sensor. touch_function is called to compute the raw output force, which is then distributed over the sensors using response_function.

The indices of the output dictionary sensor_outputs and the sensor dictionary sensor_positions are aligned, such that the i`th sensor on geom `j has position .sensor_positions[j][i] and output in .sensor_outputs[j][i].

Returns

An array containing all the touch sensations.

Return type

np.ndarray

spread_linear(contact_id, geom_id, force)

Response function. Distributes the output force linearly based on distance.

For a contact and a raw force we get all sensors within a given distance to the contact point and then distribute the force such that the force reported at a sensor decreases linearly with distance between the sensor and the contact point. Finally, the total force is normalized such that the total force over all sensors for this contact is identical to the raw force. The scaling distance is given by double the distance between sensor points.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the sensing geom.

  • force (np.ndarray) – The raw force.

nearest(contact_id, geom_id, force)

Response function. Adds the output force directly to the nearest sensor.

Parameters
  • contact_id (int) – The ID of the contact.

  • geom_id (int) – The ID of the geom.

  • force (np.ndarray) – The raw output force.

touch_function: Callable
response_function: Callable
mimoTouch.touch.scale_linear(force, distances, scale)

Used to scale forces linearly based on distance.

Adjusts the force by a simple factor, such that force falls linearly from full at distance = 0 to 0 at distance >= scale.

Parameters
  • force (np.ndarray) – The unadjusted force.

  • distances (np.ndarray) – The adjusted force reduces linearly with increasing distance.

  • scale (float) – The scaling limit. If distance >= scale the return value is reduced to 0.

Returns

The scaled force.

Return type

np.ndarray

class mimoTouch.touch.TrimeshTouch(env, touch_params)

Bases: mimoTouch.touch.Touch

A touch class with sensor meshes using MuJoCo bodies as the basic sensing component.

Sensor points are simply spread evenly over individual geoms. Geoms belonging to the same body are then merged, removing all intersecting sensors. Nearest sensors are determined through adjacency to the closest vertex, but distances are still euclidean distance instead of geodesic. For runtime reasons multiple datastructures are cached, so the sensor positions in sensor_positions should not be altered as they are tied to the underlying sensor mesh. Trimesh is used for the mesh operations. Supported output types are

  • ‘force_vector’: The contact force vector (normal and frictional forces) reported in the coordinate frame of the sensing body.

  • ‘force_vector_global’: Like ‘force_vector’, but reported in the world coordinate frame instead.

  • ‘normal_force’: Returns the normal force only, as a vector in the frame of the sensing body.

The output can be spread to nearby sensors in two different ways:

  • ‘nearest’: Directly add the output to the nearest sensor.

  • ‘spread_linear’: Spreads the force to nearby sensor points, such that it decreases linearly with distance to the contact point. The force drops to 0 at twice the sensor scale. The force per sensor is normalised such that the total force is conserved.

Touch functions return their output, while response functions do not return anything and instead write their adjusted forces directly into the output dictionary.

An LRU cache is used to speed up performance of the nearest sensor point searches. This cache persists through calls to reset().

The following attributes are provided in addition to those of Touch.

m_data

A direct reference to the MuJoCo simulation data object.

Type

mujoco.MjData

m_model

A direct reference to the MuJoCo simulation model object.

Type

mujoco.MjModel

meshes

A dictionary containing the sensor mesh objects for each body.

Type

Dict[int, trimesh.Trimesh]

active_vertices (Dict[int, np.ndarray]

A dictionary of masks. Not every sensor point will be active as they may intersect another geom on the same body. Only active vertices contribute to the output, but inactive ones are still required for mesh operations. If a sensor is active the associated entry in this dictionary will be True, otherwise False.

plotting_limits (Dict[int, float]

A convenience dictionary listing axis limits for plotting forces or sensor points for geoms.

contact_tuples

A list of tuples listing the contact index, the relevant sensing body and the raw contact forces for that contact. Note that a contact may appear twice if both involved bodies have sensors.

Type

List[Tuple[int, int, np.ndarray]]

_submeshes

A dictionary like meshes, but storing a list of the individual geom meshes instead.

Type

Dict[int, List[trimesh.Trimesh]]

_active_subvertices

A dictionary like active_vertices, but storing a list of masks for each geom mesh instead.

Type

Dict[int, List[np.ndarray]

_vertex_to_sensor_idx

A dictionary that maps the indices for each active vertex. Calculations happen on submeshes, so the indices have to mapped onto the output array. This dictionary stores that mapping.

Type

Dict[int, List[np.ndarray]]

_neighbour_cache

An LRU cache storing the results for the nearest neighbour searches. Hit rate and current size can be determined with ._neighbour_cache.hits() and ._neighbour_cache._cache.currsize respectively.

Type

LRUCache

VALID_TOUCH_TYPES = {'force_vector': 3, 'force_vector_global': 3, 'normal_force': 3}

A dictionary listing valid touch output types and their sizes.

VALID_RESPONSE_FUNCTIONS = ['nearest', 'spread_linear']

A list of valid surface response functions.

add_body(body_id=None, body_name=None, scale=math.inf)

Adds sensors to the given body.

Given a body, either by ID or name, spread sensor meshes over all geoms and adds them to the output. If both ID and name are provided the name is ignored. The distance between sensor points is determined by scale. This function has to handle all the arrays required for quick access after initialization, so it populates the submesh, mesh, mask and index mapping dictionaries. The names of bodies are determined by the scene XMLs while the IDs are assigned during compilation.

Parameters
  • body_id (int|None) – ID of the body.

  • body_name (str|None) – Name of the body. If body_id is provided this parameter is ignored!

  • scale (float) – The distance between sensor points.

sensing_bodies()

Returns the IDs of all bodies with sensors.

Returns

A list with the IDs for all bodies that have sensors.

Return type

List[int]

has_sensors(body_id)

Returns True if the body has sensors.

Parameters

body_id (int) – The ID of the body.

Returns

True if the body has sensors, False otherwise.

Return type

bool

get_sensor_count(body_id)

Returns the number of sensors for the body.

Parameters

body_id (int) – The ID of the body.

Returns

The number of sensor points for this body.

Return type

int

get_nearest_sensor(contact_pos, body_id)

Given a position in space and a body, return the sensor on the body closest to the position.

Parameters
  • contact_pos (np.ndarray) – The position. Should be a numpy array of shape (3,).

  • body_id (int) – The ID of the body. The body must have sensors!

Returns

The index of the closest sensor and the distance between contact and sensor.

Return type

Tuple[int, float]

get_k_nearest_sensors(contact_pos, body_id, k, k_margin=1.4)

Given a position and a body, find the k sensors on the body closest to the position.

Uses a cache to speed up the simulation. For a given contact we determine the closest sensor vertex on a given body. If this vertex is located in the cache, the nearest neighbour search is skipped and instead pulled from the cache. To ensure that the cache is accurate even as the contact point moves around a vertex, we store slightly more than k candidate neighbours in the cache. How many more is determined by k_margin.

Parameters
  • contact_pos (np.ndarray) – The position. Should be a numpy array of shape (3,).

  • body_id (int) – The ID of the body. The body must have sensors!

  • k (int) – The number of sensors to return.

  • k_margin (float) – The factor by which k is increased for the cache.

Returns

A tuple with two arrays, the first containing the indices of the k closest sensors and the second containing the distances between the sensors and the position.

Return type

Tuple[np.ndarray, np.ndarray]

get_sensors_within_distance(contact_pos, body_id, distance_limit, distance_margin=1.5)

Finds all sensors on a body that are within a given distance to a given contact.

The distance used is the direct euclidean distance. A sensor is included in the output if and only if:

  • It is within the distance limit to the position.

  • There is a path from the sensor to the vertex closest to the position such that all vertices on that path are also within the distance limit.

Uses a cache to speed up the simulation. For a given contact we determine the closest sensor point on a given body. If this point is located in the cache, the nearest neighbour search is skipped and instead pulled from the cache. If the point is not located in the cache we store it there. Currently, there is no pruning or limiting of the size of the cache. To facilitate accurate searches even as the contact point moves about, we search a slightly larger area on the first occurrence, which is then pruned on subsequent occurrences using the distance limit. The increase for the first search is determined by the factor distance_margin.

Parameters
  • contact_pos (np.ndarray) – The position. Should be a numpy array of shape (3,).

  • body_id (int) – The ID of the body. The body must have sensors!

  • distance_limit (float) – Sensors must be within this distance to the position to be included in the output.

  • distance_margin (float) – How much the search limit is increased on the first search. Default 1.5.

Returns

A tuple with two arrays, the first containing indices of all sensors on the body that are within distance to the contact and the second containing the distances between the sensors and the position.

Return type

Tuple[np.ndarray, np.ndarray]

get_contact_position_world(contact_id)

Get the position of a contact in the world frame.

Note that this is halfway between the touching geoms. Since geoms can intersect this point will likely be located inside both.

Parameters

contact_id (int) – The ID of the contact.

Returns

An array with the contact position.

Return type

np.ndarray

get_contact_position_relative(contact_id, body_id)

Get the position of a contact in the coordinate frame of a body.

This position is corrected for the intersection of the bodies, i.e. it will be located at the surface of the sensing body.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

Returns

An array with the contact position.

Return type

np.ndarray

get_raw_force(contact_id, body_id)

Collect the full contact force in MuJoCos own contact frame.

By convention the normal force points away from the first geom listed, so the forces are inverted if the first geom is the sensing geom.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The relevant body in the contact. One of the geoms belonging to this body must be involved in the contact!

Returns

An array with shape (3,) with the normal force and the two tangential friction forces.

Return type

np.ndarray

get_contact_normal(contact_id, body_id)

Returns the normal vector of contact (unit vector in direction of normal) in body coordinate frame.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

Returns

An array of shape (3,) containing the normal vector.

Return type

np.ndarray

force_vector_global(contact_id, body_id)

Touch function. Returns the full contact force in world frame.

Given a contact returns the full contact force, i.e. the vector sum of the normal force and the two tangential friction forces, in the world coordinate frame. The body is required to account for MuJoCo conventions and convert coordinate frames.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

Returns

An array of shape (3,) with the forces.

Return type

np.ndarray

force_vector(contact_id, body_id)

Touch function. Returns full contact force in the frame of the body.

Same as force_vector_global(), but the force is returned in the coordinate frame of the body.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

Returns

An array of shape (3,) with the forces.

Return type

np.ndarray

normal_force(contact_id, body_id)

Touch function. Returns normal force in the frame of the body.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

Returns

An array of shape (3,) with the normal force.

Return type

np.ndarray

get_contacts()

Collects all active contacts involving bodies with touch sensors.

For each active contact with a sensing geom we build a tuple (contact_id, body_id, forces), where contact_id is the ID of the contact in the MuJoCo arrays, body_id is the ID of the sensing body and forces is a numpy array of the raw output force, as determined by touch_type.

Returns

A list of tuples with contact information.

Return type

List[Tuple[int, int, np.ndarray]]

get_empty_sensor_dict(size)

Returns a dictionary with empty sensor outputs.

Creates a dictionary with an array of zeros for each body with sensors. A body with n sensors has an empty output array of shape (n, size). The output of this function is equivalent to the touch sensor output if there are no contacts.

Parameters

size (int) – The size of a single sensor output.

Returns

The dictionary of empty sensor outputs.

Return type

Dict[int, np.ndarray]

flatten_sensor_dict(sensor_dict)

Concatenates a touch output dictionary into a single large array in a deterministic fashion.

Output dictionaries list the arrays of sensor outputs for each body. This function concatenates these arrays together in a reproducible fashion to avoid key order anomalies. Bodies are sorted by their ID.

Parameters

sensor_dict (Dict[int, np.ndarray]) – The output dictionary to be concatenated.

Returns

The concatenated array.

Return type

np.ndarray

get_touch_obs()

Produces the current touch sensor outputs.

Does the full contact getting-processing process, such that we get the forces, as determined by touch_type and response_type, for each sensor. touch_function is called to compute the raw output force, which is then distributed over the sensors using response_function.

The indices of the output dictionary sensor_outputs and the sensor dictionary sensor_positions are aligned, such that the ith sensor on body has position .sensor_positions[body][i] and output in .sensor_outputs[body][i].

Returns

An array containing all the touch sensations.

Return type

np.ndarray

spread_linear(contact_id, body_id, force)

Response function. Distributes the output force linearly based on distance.

For a contact and a raw force we get all sensors within a given distance to the contact point and then distribute the force such that the force reported at a sensor decreases linearly with distance between the sensor and the contact point. Finally, the total force is normalized such that the total force over all sensors for this contact is identical to the raw force. The scaling distance is given by double the distance between sensor points.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the sensing body.

  • force (np.ndarray) – The raw force.

nearest(contact_id, body_id, force)

Response function. Adds the output force directly to the nearest sensor.

Parameters
  • contact_id (int) – The ID of the contact.

  • body_id (int) – The ID of the body.

  • force (np.ndarray) – The raw output force.

plot_sensors_body(body_id=None, body_name=None, title='')

Plots the sensor positions for a body.

Given either an ID or the name of a body, plot the positions of the sensors on that body.

Parameters
  • body_id (int|None) – The ID of the body.

  • body_name (str|None) – The name of the body. This is ignored if the ID is provided!

  • title (str) – The title of the plot. Empty by default.

plot_force_body(body_id=None, body_name=None, title='')

Plot the sensor output for a body.

Given either an ID or the name of a body, plots the positions and outputs of the sensors on that body.

Parameters
  • body_id (int|None) – The ID of the body.

  • body_name (str|None) – The name of the body. This is ignored if the ID is provided!

  • title (str) – The title of the plot. Empty by default.

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

plot_force_bodies(body_ids=[], body_names=[], title='', focus='world', show_contact_points=True)

Plot the sensor output for a list of bodies.

Given a list of bodies, either by ID or by name, plot the positions and outputs of all sensors on the bodies. The current relative positions and orientations of the bodies in the simulation are respected. The parameter focus determines how the coordinates are centered. Two options exist: - ‘world’: In this setting all the coordinates are translated into global coordinates - ‘first’: In this setting all the coordinates are translated into the frame of the first body in the list.

Parameters
  • body_ids (List[int]) – A list of IDs of the bodies that should be plotted.

  • body_names (List[str]) – A list of the names of the bodies that should be plotted. This is ignored if body_ids is provided!

  • title (str) – The title of the plot. Empty by default.

  • focus (str) – Coordinates are moved into a consistent reference frame. This parameter determines that reference frame. Must be one of ["world", "first"]. Default “world”.

  • show_contact_points (bool) – If True the actual contact points are also plotted. Note that these are corrected for intersecting bodies. Default True.

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

plot_force_body_subtree(body_id=None, body_name=None, title='', show_contact_points=False)

Plot the sensor output for the kinematic subtree with the given body at its root.

Given a body, collects all descendant bodies in the kinematic tree and plot the positions and outputs of their sensors. The current relative positions and orientations of the bodies in the simulation are respected and all coordinates are moved into the coordinate frame of the root body.

Parameters
  • body_id (int|None) – The ID of the root body for the subtree. Either this or body_name must be supplied.

  • body_name (str|None) – The name of the root body. Either this or body_id must be supplied. If both are provided, body_name is ignored.

  • title (str) – The title of the plot. Empty by default.

  • show_contact_points (bool) – If True, the actual rigid body contact points are also plotted. Default False.

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

visualize_contacts_subtree(root_id=None, root_name=None, show_contact_points=False, focus_body=None, camera_offset=None)

Generates a neat visualization of parts of MIMo and their contact forces.

Starting from a root body, renders the child parts of MIMo and the contact sensors on them. Contact forces are shown by coloring the sensor points and increasing their size corresponding to the contact force. Unlike the other force plotting functions this one only visualized the magnitude of the contact force, not the direction.

For convenience the render can be centered on a body, with an additional offset for the camera. This allows controlling the camera placement.

Parameters
  • root_id (int|None) – The id of the root body. Either this or root_name must be supplied.

  • root_name (str|None) – The name of the root body. Either this or root_id must be supplied. If both are provided, root_name is ignored.

  • show_contact_points (bool) – Whether to render the MuJoCo point contacts. Default False.

  • focus_body (str|None) – The body, by name, on which the render will be centered. If None, the bodies will be at their coordinates as in the scene.

  • camera_offset (np.ndarray|None) – An array with a camera position offset. This allows moving the camera relative to the bodies. Must have shape (3,). If None, there is no offset. Default None.

Returns

A tuple (fig, ax) with the pyplot figure and axis objects.

Return type

Tuple[plt.Figure, plt.Axes]

touch_function: Callable
response_function: Callable

mimoTouch.sensorpoints

This module contains functions for distributing points over geometric primitives.

mimoTouch.sensorpoints.spread_points_box(resolution, sizes, return_normals=False)

Spreads points over the surface of a box.

Spreads points over the surface of a box such that the points are spaced at approximately resolution distance to each other. The box is subdivided into an integer number of segments. If the box is too small for the resolution a single point at the center of the box is returned instead. The box is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • sizes (np.ndarray) – The size of the box. Should be a numpy array of shape (3,).

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensorpoints.spread_points_sphere(resolution, radius, return_normals=False)

Spreads points over the surface of a sphere.

Spreads points over the surface of a sphere such that the points are spaced at approximately resolution distance to each other. This is done by splitting the sphere into latitude rings and spreading a variable number of points over each ring. If the sphere is too small for the resolution a single point at the center of the sphere is returned instead. The sphere is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • radius (float) – The radius of the sphere.

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensorpoints.spread_points_ellipsoid(resolution, radii, return_normals=False)

Spreads points over the surface of an ellipsoid.

Spreads points over the surface of an ellipsoid. This is done by spreading points over a sphere and then projecting those onto the ellipsoid. As a result the distance between points varies more strongly the less spherical the ellipsoid gets. If the ellipsoid is too small for the resolution a single point at the center is returned instead. The ellipsoid is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • radii (np.ndarray) – An array of shape (3,) containing the radii for the three axis.

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensorpoints.spread_points_pipe(resolution: float, length: float, radius: float, return_normals: bool = False)

Spreads points over the surface of a pipe.

Spreads points over the surface of a pipe, with no caps, such that the distance between points is approximately resolution. This is done by subdividing the cylinder into an integer number of pie segments. If the pipe is too small for the resolution a single point at the center is returned instead. The pipe is centered on (0,0,0) with the longitudinal axis aligned with the z-axis.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the pipe.

  • radius (float) – The radius of the pipe.

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensorpoints.spread_points_cylinder(resolution, length, radius, return_normals=False)

Spreads points over the surface of a cylinder.

Spreads points over the surface of a cylinder, such that the distance between points is approximately resolution. This is done by subdividing the cylinder into an integer number of pie segments. The caps are split into rings of different radii with points spread over each ring. If the cylinder is too small for the resolution a single point at the center is returned instead. The cylinder is centered on (0,0,0) with the longitudinal axis aligned with the z-axis.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the cylinder.

  • radius (float) – The radius of the cylinder.

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensorpoints.spread_points_capsule(resolution, length, radius, return_normals=False)

Spreads points over the surface of a capsule.

A capsule is a cylinder with a hemisphere capping each end. Spreads points over the surface of a capsule, such that the distance between points is approximately resolution. This is done by subdividing the capsule into an integer number of pie segments. The caps are split into rings of latitude with points spread over each ring. If the capsule is too small for the resolution a single point at the center is returned instead. The capsule is centered on (0,0,0) with the longitudinal axis aligned with the z-axis.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the cylindrical section.

  • radius (float) – The radius of the cylinder and hemispheres.

  • return_normals (bool) – If True we return the normal vector of each point in addition to the position. Default False.

Returns

Always returns an array of shape (n, 3) with the positions of sensor points. If return_normals is True, additionally returns a second array with normal vectors for every point.

Return type

Union[Tuple[np.ndarray, np.ndarray], np.ndarray]

mimoTouch.sensormeshes

This module contains functions for distributing points over geometric primitives.

The functions here function much like those in mimoTouch.sensorpoints, but the points are structured as a watertight mesh instead of being a set of points.

mimoTouch.sensormeshes.mesh_box(resolution, sizes)

Spreads a mesh over the surface of a box.

Spreads a mesh over the surface of a box such that the points are spaced at approximately resolution distance to each other. The box is subdivided into an integer number of segments. If the box is too small for the resolution a single point at the center of the box is returned instead. The box is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • sizes (np.ndarray) – The size of the box. Should be a numpy array of shape (3,).

Returns

The output mesh.

Return type

trimesh.Trimesh

mimoTouch.sensormeshes.mesh_sphere(resolution, radius)

Spreads a mesh over the surface of a sphere.

Spreads a mesh over the surface of a sphere such that the points are spaced at approximately resolution distance to each other. This is done by splitting the sphere into latitude rings and spreading a variable number of points over each ring. The points are meshified by taking the convex hull over the points. If the sphere is too small for the resolution a single point at the center of the sphere is returned instead. The sphere is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • radius (float) – The radius of the sphere.

Returns

The output mesh.

Return type

trimesh.Trimesh

mimoTouch.sensormeshes.mesh_ellipsoid(resolution, radii)

Spreads a mesh over the surface of an ellipsoid.

Spreads points over the surface of an ellipsoid. This is done by spreading points over a sphere and then projecting them onto the ellipsoid. Finally, the points are turned into a watertight mesh by taking the convex hull over the points. Since the points are projected the distance between points varies more strongly the less spherical the ellipsoid gets. If the ellipsoid is too small for the resolution a single point at the center is returned instead. The ellipsoid is centered on (0,0,0).

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • radii (np.ndarray) – An array of shape (3,) containing the radii for the three axis.

Returns

The output mesh.

Return type

trimesh.Trimesh

mimoTouch.sensormeshes.mesh_pipe(resolution, length, radius)

Spreads a mesh over the surface of a pipe.

Spreads points over the surface of a pipe, with no caps, such that the distance between points is approximately resolution. This is done by subdividing the cylinder into an integer number of pie segments. If the pipe is too small for the resolution a single point at the center is returned instead. The pipe is centered on (0,0,0) with the longitudinal axis aligned with the z-axis. The mesh produced by this function is not watertight, since it is missing caps.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the pipe.

  • radius (float) – The radius of the pipe.

Returns

The output mesh.

Return type

trimesh.Trimesh

mimoTouch.sensormeshes.mesh_cylinder(resolution, length, radius)

Spreads a mesh over the surface of a cylinder.

Spreads points over the surface of a cylinder, such that the distance between points is approximately resolution. This is done by subdividing the cylinder into an integer number of pie segments. The caps are split into rings of different radii with points spread over each ring. The cylindrical section is triangulated manually while the caps are processed with the Delaunay triangulation from scipy. If the cylinder is too small for the resolution a single point at the center is returned instead. The cylinder is centered on (0,0,0) with the longitudinal axis aligned with the z-axis.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the cylinder.

  • radius (float) – The radius of the cylinder.

Returns

The output mesh.

Return type

trimesh.Trimesh

mimoTouch.sensormeshes.mesh_capsule(resolution, length, radius)

Spreads a mesh over the surface of a capsule.

A capsule is a cylinder with a hemisphere capping each end. Spreads points over the surface of a capsule, such that the distance between points is approximately resolution. This is done by subdividing the capsule into an integer number of pie segments. The caps are split into rings of latitude with points spread over each ring. If the capsule is too small for the resolution a single point at the center is returned instead. The capsule is centered on (0,0,0) with the longitudinal axis aligned with the z-axis.

Parameters
  • resolution (float) – Approximate distance between neighbouring points on the surface.

  • length (float) – The length of the cylindrical section.

  • radius (float) – The radius of the cylinder and hemispheres.

Returns

The created mesh.

Return type

trimesh.Trimesh