Skip to main content

ICON C++ client API

This guide describes the ICON C++ client API.

Introduction​

ICON is a real-time control framework, intended for industrial robot applications.

The client API lets you control ICON-compatible hardware, which may be real or simulated.

You can use the client API to command a compatible robot arm to perform basic motions, such as point-to-point moves in joint space, and linear moves in Cartesian space. Such motions are known as "actions" in the API.

You can also command sequences of motions, as well as state machines of motions, which are executed in hard real-time. This is done by creating several pending actions and chaining them together with transitions, which are known as "reactions" in the API.

Namespaces​

Most symbol names are under the ::xfa::icon:: namespace. For the purpose of brevity in this document, namespaces are dropped. To import ICON client API symbols into the current namespace:

using xfa::icon::Channel;
using xfa::icon::Client;
using xfa::icon::ConnectionParams;
using xfa::icon::Session;
// etc...

Connect to the server with a Channel​

The ICON client is initialized by connecting to an ICON server at a specified gRPC address using an intrinsic.util.grpc.connection.ConnectionParams struct.

The instance name of the robot controller resource is usually provided by an external component, like the skills interface. Its default value is 'robot_controller'.

See Connecting to Services exposed by Assets for detailed information and examples.

Sessions and Clients​

The client API's methods are split between the session class and the client class. Methods relating to motion, such as describing and executing actions, require a session. Other methods are part of the client class.

Construct a client class by passing in the channel.

  // Construct an ICON client object.
  Client client(channel);

Session initialization is discussed in the Sessions section.

Operational state​

The ICON server is always in one of the following operational states:

  • DISABLED: Indicates that motion is disabled and sessions may not be started.
  • FAULTED: Indicates that the robot or server is in an erroneous state.
  • ENABLED: Indicates that motion is enabled and session may be started.

The OperationalState is part of the OperationalStatus proto.

To check the current operational state:

  absl::StatusOr<OperationalStatus> operational_status =
      icon_client.GetOperationalStatus();
  if (!operational_status.ok()) {
    // ... error handling ...
  }

  std::cout << "Operational state is: "
            << *operational_state << std::endl;

  switch (operational_status->state()) {
    case OperationalState::kDisabled:
      // ...
      break;
    case OperationalState::kFaulted:
      std::cout << "Fault reason:" << operational_status->fault_reason();
      // ...
      break;
    case OperationalState::kEnabled:
      // ...
      break;
    default:
      // Unknown state ...
  }

To check for particular operational states, you can use the convenience functions IsDisabled(operational_status), IsFaulted(operational_status) and IsEnabled(operational_status).

To put the server into the ENABLED state from the DISABLED state:

  absl::Status result = icon_client.Enable();
  if (!result.ok()) {
    // ... error handling ...
  }

Disabling the server will immediately end any running sessions, even those created by other clients. To put the server into the DISABLED state from the ENABLED state:

  absl::Status result = icon_client.Disable();
  if (!result.ok()) {
    // ... error handling ...
  }

To put the server into the DISABLED state from the FAULTED state:

  absl::Status result = icon_client.ClearFaults();
  if (!result.ok()) {
    // ... error handling ...
  }

Parts​

A part represents a piece of hardware, either real or simulated, that the server can control. From the client's perspective, a part has:

  • A string name identifier, e.g. 'arm'.
  • A current status, which is updated every real-time cycle on the server, and which can be queried from the client.
  • A read-only configuration, which is static over the lifetime of the ICON server and contains:
    • A list of the feature interfaces the part supports.
    • A GenericPartConfig message with config information for each supported feature interface.
    • Optionally, a proto.Any message for more specialized configuration data. There are two additional fields, config_message_type and config_descriptor_set, which can be used to unpack the proto.Any message, even if an application is not otherwise aware of the message type contained within.

Listing Parts​

To list the available parts:

  absl::StatusOr<std::vector<std::string>> parts = icon_client.ListParts();
  if (!parts.ok()) {
    // ... error handling ...
  }
  std::cout << "Part names: " << absl::StrJoin(parts, ", ") << std::endl;

Getting Part Configuration​

To get generic config information (like the number of joints or the limits of a part), use:

// Extract the number of joints for part "arm".
absl::StatusOr<RobotConfig> robot_config = icon_client.GetConfig();
if (!robot_config.ok()) {
  // error handling
}

absl::StatusOr<::intrinsic_proto::icon::GenericPartConfig> part_config =
                  robot_config.GetGenericPartConfig("arm");
if (!part_config.has_joint_position_config()) {
  return absl::FailedPreconditionError(
      "Part 'arm' does not have JointPosition configuration.");
}
const size_t part_ndof = part_config.joint_position_config().num_joints();

Specialized Part Configuration​

warning

Unless your client code is very specialized towards a certain kind of hardware, you likely will not need this.

The configuration message also contains part-specific configuration that does not fit into the generic_config field, stored as a proto.Any.

An application that expects to handle a specific part implementation can attempt to unpack the proto.Any configuration data and throw a run-time error if its message type is not the one it expects. Use RobotConfig::GetPartConfigAny or RobotConfig::GetPartConfig<ProtoT>() to extract this message.

For introspection, an application may also use the config_message_type and config_descriptor_set members of the message to unpack the proto.Any message, even if it is not otherwise aware of the message type contained within.

This is useful for generic introspection tools, like those that print a human-readable representation of the configuration.

Getting Part Status​

To get a snapshot of a part's current status, use:

  // Example of getting the current joint angles for a part.
  absl::StatusOr<RobotStatus> robot_status = icon_client.GetStatus();
  if (!robot_status.ok()) {
    // ... Error handling ...
  }
  absl::StatusOr<proto::PartStatus> part_status =
      robot_status->GetPartStatus(part_name);
  if (!part_status.ok()) {
    // ... Error handling (Part not found) ...
  }
  std::cout << "Sensed joint positions:" << std::endl;
  for (const proto::PartJointState& joint_state : part_status.joint_states()) {
    std::cout << joint_state.position_sensed() << std::endl;
  }

Sessions​

To command motion on a part, you must first start a session. A session:

  1. Claims exclusive control over one or more parts. By doing so, it prevents other client applications from moving those parts while we are controlling them.
  2. Scopes action instance IDs.
  3. Bounds the lifetime of action-related server-side objects, including actions and reactions.
  4. Manages action-related event handling. In the ICON C++ client library, this is done by running a "watcher loop".

To start a session, provide a channel and a list of part names to take control over.

  absl::StatusOr<std::unique_ptr<Session>> session =
      Session::Start(icon_channel, {part_name_0, part_name_1});
  if (!session.ok()) {
    // ... Error handling ...
  }

The session ends when the session object's unique pointer goes out-of-scope.

Actions​

An action describes how the server should control one or more parts. Each action corresponds to real-time cyclic control logic that runs on the server.

An action instance is a specific instantiation that controls one or more parts.

From the client's perspective, an action instance has:

  • An action instance ID, which is a session-scoped user-assigned integer.
  • A string action type name, specifying the action type, e.g. "intrinsic.point_to_point_move".
  • Action-specific fixed parameters, represented as a proto.Any, which are specified when creating the instance.
  • Action-specific streaming inputs, which can be used to stream data to a running instance. The payloads are represented as proto.Any objects.
  • Action-specific real-time signals, which can be used to signal an event in real-time using Reactions.
  • A list of reactions, which may reference action-specific state variables in their condition expressions.

Action Signatures​

An action signature is the metadata describing a type of action available on the server. It includes the specific types of all fixed parameters, streaming inputs and state variables.

To get the signatures of all available actions:

  // Example that prints available action type names.
  absl::StatusOr<std::vector<proto::ActionSignature>> action_signatures =
      ListActionSignatures();
  if (!action_signatures.ok()) {
    // ... Error handling ...
  }
  std::cout << "Available Action Types: " << std::endl;
  for (const proto::ActionSignature& action_signature : *action_signatures) {
    std::cout << action_signature.action_type_name() << std::endl;
  }

Action Instances​

To command a part (or multiple parts), an action instance is created and then started. Here is an example of creating an action instance that performs a point-to-point move:

First, create an ActionDescriptor describing the action instance's parameters:

  std::vector<double> goal = {0.1, 0.2, 0.3, 0.4, 0.5, 0.6};
  std::vector<double> zero(6);
  PointToPointMoveInfo::FixedParams params;
  *params.mutable_goal_position()->mutable_joints() = {goal.begin(),
                                                       goal.end()};
  *params.mutable_goal_velocity()->mutable_joints() = {zero.begin(),
                                                       zero.end()};
  constexpr ActionInstanceId kActionId(0);
  constexpr ReactionHandle kDoneHandle(0);
  ActionDescriptor joint_move_descriptor =
      ActionDescriptor(PointToPointMoveInfo::kActionTypeName, kActionId,
                       Part("arm"))
          .WithFixedParams(params)
          .WithReaction(ReactionDescriptor(IsDone()).WithHandle(kDoneHandle));

Next, create an action instance on the server from this ActionDescriptor:

  absl::StatusOr<Action> joint_move = session->AddAction(joint_move_descriptor);
  if (!joint_move.ok()) {
    // ... Error handling ...
  }

Then start the action to begin the motion:

  absl::Status status = session->StartAction(joint_move);
  if (!status.ok()) {
    // ... Error handling ...
  }

Now wait for the action to finish:

  if (!session->RunWatcherLoopUntilReaction(kDoneHandle).ok()) {
    // ... error handling ...
  }

Action instances are long-running and remain active on their parts until either:

  1. The action instance is preempted by calling session->StartAction(...) with any of the same parts, or
  2. The action instance is preempted in response to a reaction triggering, or
  3. The session ends, in which case the parts come to a stop.

Reactions​

Reactions are an essential and powerful feature of ICON, enabling real-time and non-real time responses to events that occur in the real-time system.

A reaction has:

  • A condition that is evaluated every control cycle by the server.
  • A list of responses that describe what should happen if and when the condition is satisfied.

A reaction is associated with an action instance. The reaction's condition is only evaluated (and responses will only trigger) while the associated action instance is active.

A reaction's responses are triggered at most once for an active action instance. If the action instance is restarted, the responses may trigger again.

Conditions​

A reaction's condition is an expression that references action-specific state variables.

Here are some examples:

  // Note: the meaning of is_done() is action-specific, and most
  // feedback-controlled actions are never done.
  Condition cond0 = IsDone();

  cond1 = IsLessThan("xfa.distance_to_sensed ", 0.25);
  cond2 = IsGreaterThanOrEqual("xfa.action_elapsed_time", 8.0);
  cond3 = IsApprox("xfa.distance_to_sensed_linear", 0.25);
  cond4 = IsTrue('xfa.is_stopped');
  cond5 = AnyOf({IsGreaterThan("xfa.action_elapsed_time", 10.0),
                 IsLessThan("xfa.distance_to_sensed_linear", 0.25)});
  cond6 = AllOf({IsDone(), IsGreaterThan("xfa.action_elapsed_time", 3.00)});

It is also possible to specify conditions on fields of a part status (e.g. joint position sensors of the arm part or forces of the force torque sensor part). The preceding examples used strings directly for creating conditions with action state variables. However, the ICON API offers builder functions to create the correct string (called state variable path) for a part status condition. A part status condition is a Boolean expression over one or more state variable paths. A state variable path is a reference to a field on the part status (e.g., position of joint 3) or a computed state variable from a vector in the part status (e.g., the force magnitude of a force sensor).

Here are some examples:

// Is fulfilled when the joint position at index 0 is greater than 0.5 radians.
Condition position_cond = IsGreaterThan(
                     ArmSensedPositionStateVariablePath(arm_part_name, 0), 0.5);

// Is fulfilled when the magnitude of the x, y, z forces measured with the
// force torque sensor of the part with name `ft_part_name` is less than 0.5
// Newton.
Condition force_cond = IsLessThan(
                     FTForceMagnitudeAtTipStateVariablePath(ft_part_name), 0.5);

All C++ builder functions are available in intrinsic/icon/cc_client/state_variable_path.h.

Responses​

A reaction's response describes what should happen when the reaction's condition is satisfied. This may be:

  • A real-time response:
    • reaction_descriptor.WithRealtimeActionOnCondition(...): Start another action on the server, the next control cycle.
    • reaction_descriptor.WithRealtimeSignalOnCondition(...): Triggers a real-time signal which can be read within the associated action on the next control cycle.
  • A client-side response:
    • reaction_descriptor.WithWatcherOnCondition(...): Triggers a client-side callback.
    • reaction_descriptor.WithHandle(...): Signal that a reaction has occurred, which can be used to unblock a waiting client.

The client-side responses require a running watcher loop in order to trigger.

Watcher Loop​

The watcher loop processes client-side reaction events for a session. Run the watcher loop with:

  if (!session->RunWatcherLoop().ok()) {
    // ... Error handling ...
  }

The Watcher Loop blocks the calling thread until session->QuitWatcherLoop() is called or the session ends. All associated callbacks are invoked on the calling thread.

The Watcher Loop can also be run using:

  if (!session->RunWatcherLoopUntilReaction(reaction_handle).ok()) {
    // ... Error handling ...
  }

In this case, the Watcher Loop will additionally return as soon as reaction_handle is triggered.

To explicitly stop the watcher loop, call:

  session->QuitWatcherLoop();

This may be called from another thread, or from a callback registered with ReactionDescriptor::WithWatcherOnCondition(...).

Waiting for an Action to Finish​

To block the client, waiting for an action to finish:

  constexpr ReactionHandle kDoneHandle(0);
  absl::StatusOr<Action> action = session->AddAction(
      ActionDescriptor(PointToPointMoveInfo::kActionTypeName,
                        ActionInstanceId(0), parts)
           .WithFixedParams(fixed_params)
           .WithReaction(
               ReactionDescriptor(IsDone()).WithHandle(kDoneHandle)));
  if (!action.ok()) {
    // ... error handling ..
  }
  if (!session->StartAction(*action).ok()) {
    // ... error handling ...
  }
  // Block, processing events, until the Action finishes:
  if (!session->RunWatcherLoopUntilReaction(kDoneHandle)) {
    // ... error handling ...
  }

Triggering a user callback​

To trigger a user callback in response to a reaction event:

  constexpr ActionInstanceId kActionId(0);
  absl::StatusOr<Action> action = session->AddAction(
      ActionDescriptor(PointToPointMoveInfo::kActionTypeName, kActionId, parts)
           .WithFixedParams(fixed_params)
           .WithReaction(ReactionDescriptor(IsDone()).WithWatcherOnCondition(
               [&session]() {
                 std::cout << "Action finished!" << std::endl;
                 session->QuitWatcherLoop();
               })));
  if (!action.ok()) {
    // ... error handling ..
  }
  if (!session->StartAction(action).ok()) {
    // ... error handling ...
  }
  // Block, processing events, until the action finishes:
  if (!session->RunWatcherLoop()) {
    // ... error handling ...
  }

Chaining Actions in Real-Time​

Real-time responses can be used to chain actions in real-time. This can be used to achieve deterministic timing of sequences of actions. Multiple actions be added at once using session->AddActions(...).

  ActionDescriptor action0 =
      ActionDescriptor(PointToPointMoveInfo::kActionTypeName,
                       ActionInstanceId(1), part_group)
          .WithFixedParams(fixed_params)
          .WithReaction(
              ReactionDescriptor(IsDone()).WithRealtimeActionOnCondition(
                  ActionInstanceId(2)));
  constexpr ReactionHandle kSequenceDone(0);
  ActionDescriptor action1 =
      ActionDescriptor(kStopAction, ActionInstanceId(2), part_group)
          .WithReaction(ReactionDescriptor(IsDone()).WithHandle(kSequenceDone));

  absl::StatusOr<std::vector<Action>> actions =
      session->AddActions({jmove, jstop});
  if (!actions.ok()) {
    // ... error handling ..
  }
  // Start the sequence action0 => action1.
  if (!session->StartAction(action.front()).ok()) {
    // ... error handling ...
  }
  // Wait for the sequence to end.
  if (!session->RunWatcherLoopUntilReaction(kSequenceDone)) {
    // ... error handling ...
  }

ICON Examples​

Prerequisites​

Please make sure that the robot is enabled.

note

If you are connecting over ssh, you will need to append -- --server=localhost:17080 in the commands below, e.g. bazel run //intrinsic/icon/examples:joint_move -- --server=localhost:17080

caution

If your ICON (real-time control service) instance name is different from the default of robot_controller, you need to adapt the --instance argument in the commands below.

A wrong instance argument leads to an error like: Check failed: Run(intrinsic::ConnectionParams::ResourceInstance( absl::GetFlag(FLAGS_instance), absl::GetFlag(FLAGS_server))) is OK (UNIMPLEMENTED: )

Joint move example​

This example creates three actions, a point-to-point joint move to goal1, a stop motion (deceleration to zero velocity), and a point-to-point joint move to goal2. Each action has a real time reaction to switch to the next action on kIsDone, which means the nominal joint position and velocity are reached in the trajectory generator of that action. The last action sends a callback to end the session.

bazel run //intrinsic/icon/examples:joint_move -- --instance=robot_controller

Cartesian move example​

Run an example program that executes both a joint space and a short Cartesian motion (in positive x direction).

bazel run //intrinsic/icon/examples:joint_then_cart_move -- --instance=robot_controller

Details​

Open the source code of the example using:

gedit intrinsic/icon/examples/joint_then_cart_move.cc

This example first creates an ICON Channel and then calls the example JointThenCartMove implementation.

XFA_ASSIGN_OR_RETURN(auto icon_channel,
                     xfa::icon::Channel::Make(server, instance));

return JointThenCartMove(part_name, icon_channel);

Open the source code of the example JointThenCartMove using:

gedit intrinsic/icon/examples/joint_then_cart_move_lib.cc

In this method we first create an ICON Client and Session.

  xfa::icon::Client client(icon_channel);

  XFA_ASSIGN_OR_RETURN(
      std::unique_ptr<xfa::icon::Session> session,
      xfa::icon::Session::Start(icon_channel, {part_name}));

To move the robot to a known pose, it adds and executes a point-to-point joint move Action.

Adding an Action in ICON is done by filling a xfa::icon::ActionDescriptor object with the name of the Action, as well as any parameters and reactions:

  xfa::icon::ActionDescriptor move_to_start =
      xfa::icon::ActionDescriptor(
          xfa::icon::PointToPointMoveInfo::kActionTypeName, kJointMoveToStartId,
          part_name)
          .WithFixedParams(xfa::icon::CreatePointToPointMoveFixedParams(
              initial_joint_pos, zero_velocity))
          .WithReaction(
                  xfa::icon::ReactionDescriptor(xfa::icon::IsDone())
                  // This registers a callback that will be called once the
                  // condition becomes true. The callback runs in a separate,
                  // non-realtime thread.
                  // It's important to have a callback that invokes the
                  // QuitWatcherLoop() method, because otherwise the main thread
                  // will block on RunWatcherLoop() forever.
                  .WithWatcherOnCondition([&session]() {
                    std::cout << "Reached Start Position." << std::endl;
                    session->QuitWatcherLoop();
                  }));

Next, the program defines a stop action to actively hold the robot in the start position. Both actions are added to a session which is then started. Starting the session moves the robot.

Next, the program reads the current pose of the robot in Cartesian space by using ICON's GetSinglePartStatus() method.

Finally, it adds two more actions to the Session:

  • A second point-to-point joint move to a position that's different from the first. This has a real time response that switches to the following action, and a non-real time response invokes a callback to print a log statement.
  • A linear cartesian motion back to where the robot was after the first point-to-point move above. This has a non-realtime response that invokes a callback that ends the ICON session.

Unlike the non-real time response for the first Action, a real time response takes effect on the realtime ICON server, rather than in the non-real time client application, and does so in the very next control cycle after the condition has become true. Realtime reactions are limited to changing the active Action, and triggering real time signals as opposed to the arbitrary callbacks for non-real time reaction responses.

Introspection example​

This example shows the introspection methods offered by the ICON API.

It first lists the available Parts, then the available Actions (including the compatible Parts for each), and finally the current PartStatus for each Part. Run it like this:

bazel run //intrinsic/icon/examples:introspection -- --instance=robot_controller | less

Scroll through the output using the arrow keys, and exit less by pressing q.

These introspection methods allow a program to determine at runtime which Parts it needs to use to execute a given Action.

For example, a program that wants to move an arm in Cartesian space does not need to explicitly know the name of the corresponding Part. It can simply check the list of available Actions. If the xfa.cartesian_position Action is available, it can then query the ICON server for a list of compatible Parts, and use one of those.

Command-line tools​

Some basic command-line tools, along with their source code, are provided for reference.

Reset the simulation​

The following command resets the simulated robot back to its default starting position:

bazel run //intrinsic/icon/release:reset_simulation

Clear Faults​

To clear faults on the simulated robot, run:

bazel run //intrinsic/icon/tools:clear_faults -- --instance=robot_controller --print_fault_reason

List ICON actions​

To list the Actions that are available to ICON, run:

bazel run //intrinsic/icon/tools:list_actions -- --instance=robot_controller

The output will resemble:

intrinsic.cartesian_position
intrinsic.cartesian_rangefinder_path_action
intrinsic.point_to_point_move
intrinsic.stop

See the ICON actions reference for details of using these actions. See the ICON actions reference for details of using these actions.