Implement agent animations

How it works

The official AI Real-Time Interaction demo uses After Effects to design agent avatar animations and exports them as Lottie resource files. The client integrates a Lottie component library to play and control the animations. For more information about Lottie, see LottieFiles.

In the official demo, After Effects animations are split into modular components: head, hand, eyes, and covering eyes. Each module's animation resources are exported separately. The complete Lottie resource structure is as follows:

├── Avatar                       // Root directory for avatar animation resources
│   ├── bg.png                   // Background image
│   ├── Enter                    // Intro animation (while the call is connecting)
│   ├── Head                     // Head animation
│   ├── Hand                     // Hand animation
│   ├── CoveringEyes             // Hand animation for covering eyes (when the agent is interrupted)
│   ├── EyeEmotions              // Eye animations
│       ├── Interrupting         // Eye animation - Interrupted
│       ├── Listening            // Eye animation - Listening
│       ├── Thinking             // Eye animation - Thinking
│       ├── Speaking             // Eye animation - Speaking
│       ├── Happy                // Eye animation - Speaking happily
│       ├── Sad                  // Eye animation - Speaking sadly

Reference

The animation source code in the official demo is available at the following locations:

Implementation details

In the official demo, the agent's head (nodding motion) and hand animations loop continuously. When the agent's state or emotion changes, the corresponding eye animation loops. When the agent is interrupted, the "covering eyes" hand animation plays once and returns to the previous state. The following logic requires special handling:

1. As the head animation loops up and down, the eyes follow the movement. Listen to the head animation's playback progress to estimate the Y-axis offset, then update the eye animation's position in real time.

   // Listen for update events on the head animation.
   headAnimator.addAnimatorUpdateListener(new ValueAnimator.AnimatorUpdateListener() {
       @Override
       public void onAnimationUpdate(ValueAnimator animation) {
           float animatedFraction = animation.getAnimatedFraction();
           float speed = headAnimator.getSpeed();
           float progress;
           if (speed < 0) {
               progress = 1 - animatedFraction;
           } else {
               progress = animatedFraction;
           }
           // Estimate the offset based on the playback progress.
           float y = eyeOffset * (progress - 1.0f);
           // Update the eye animation's Y translation.
           eyeAnimatorContainerView.setTranslationY(y);
       }
   });

2. Define the eye animation types and handle the transition logic:

   • The Error state is final and cannot be transitioned from.

   • If the next state is Interrupting, it cannot be overridden.

   • While in the Interrupting state, the animation cannot be switched. The next state is cached and will play after the interruption animation finishes.

   • When the agent is speaking with an emotion (HappySpeaking or SadSpeaking), it cannot be switched to the neutral Speaking state.

   • Transitions to Interrupting or Error are queued and execute after the current animation cycle finishes.

   public enum EyeAnimator {
       None,           // No animation
       Start,          // Agent starting up
       Error,          // Agent error state
       Interrupting,   // Agent is interrupted
       Thinking,       // Agent is thinking or loading
       Listening,      // Agent is listening
       Speaking,       // Agent is speaking (neutral)
       HappySpeaking,  // Agent is speaking (happy)
       SadSpeaking     // Agent is speaking (sad)
   }
   
   public void setEyeAnimatorType(EyeAnimator type) {
       Log.d("Animator", "Will Set Animator Type Curr: " + currEyeAnimatorType + " To: " + type);
       if (type == EyeAnimator.None) {
           return;
       }
   
       if (currEyeAnimatorType == type) {
           return;
       }
   
       if (currEyeAnimatorType == EyeAnimator.Error) {
           // The Error state is final and cannot be transitioned from.
           return;
       }
   
       if (nextEyeAnimatorType == EyeAnimator.Interrupting || nextEyeAnimatorType == EyeAnimator.Error) {
           // If the next state is Interrupting, it cannot be overridden.
           return;
       }
       if (currEyeAnimatorType == EyeAnimator.Interrupting) {
           // While in the Interrupting state, the animation cannot be switched. The next state is cached and will play after the interruption animation finishes.
           if (type == EyeAnimator.Speaking && nextEyeAnimatorType.ordinal() > type.ordinal()) {
               // If the next state is an emotional speaking state, it cannot be switched to the neutral Speaking state.
               return;
           }
           nextEyeAnimatorType = type;
           return;
       }
   
       if (type == EyeAnimator.Speaking && currEyeAnimatorType.ordinal() > type.ordinal()) {
           // If the agent is speaking with an emotion, it cannot be switched to the neutral Speaking state.
           return;
       }
   
       if (type == EyeAnimator.Interrupting || type == EyeAnimator.Error) {
           // If the next state is Interrupting or Error, wait for the current animation to complete.
           nextEyeAnimatorType = type;
           return;
       }
       Log.d("Animator", "Set Animator Type Next: " + type + " Curr: " + currEyeAnimatorType);
       startNextEyeAnimator(type);
   }

3. When an interruption event is triggered, the "covering eyes" animation must be synchronized with the head animation — it must start at the first frame (progress=0) of the head animation's cycle. During this time, the eye animation cannot switch to another state.

   // Listen for the end of the head animation.
   headAnimator.addAnimatorListener(new AnimatorListenerAdapter() {
       @Override
       public void onAnimationEnd(Animator animation) {
           headAnimator.reverseAnimationSpeed();
           headAnimator.playAnimation();  // Loop the head animation by playing it again.
           if (nextEyeAnimatorType != EyeAnimator.None) { // Check for a queued animation type.
               startNextEyeAnimator(nextEyeAnimatorType); // Synchronously start the next animation (e.g., covering eyes).
               if (currEyeAnimatorType == EyeAnimator.Interrupting) {
                   nextEyeAnimatorType = EyeAnimator.Listening;
               } else {
                   nextEyeAnimatorType = EyeAnimator.None;
               }
           }
       }
   });

4. Avatar animations may rely on rasterized assets (bitmaps) rather than pure vector graphics, which can increase memory consumption. To reduce memory usage, export only a one-way animation (e.g., nodding down) from After Effects and use the AutoReverse mode during playback to create a loop.

Implement custom agent animations