# Prerequisite Knowledge

## ArUco

ArUco is a kind of encoding code similar to QR code. It’s commonly used in Augmented Reality.

## Camera Calibration

### Intrinsic Parameter

• focal length of the camera $f_x$ and $f_y$
• the optical center in the image $(c_x, c_y)$
• the radial distortion parameters $\gamma$

## PnP Algorithm

The PnP (Perspective n-Point) algorithm takes n points in the 3D world and their corresponding 2D points in the image as inputs, and then calculates the pose and position of the camera relative to the world. (NOTE: Chosen points cannot be colinear)

RANSAC might be useful for removing outliers.

where

• $s$ is a scale factor for the image point
• $p_c$ is coordinate in camera referece frame
• $p_w$ is coordinate in world reference frame
• $R$ is rotation matrix
• $T$ is translation vector
• $f_x$ and $f_y$ are the scaled focal lengths
• $\gamma$ is the skew parameter which is sometimes assumed to be 0
• $(c_x, c_y)$ is the principal point

# System Design

• 4 ArUcos
• 1 camera
• 1 Manipulator(UR5)

I used 4 ArUcos just for averaging the result. The algorithm will work if it’s just one ArUco. The 4 corner points of a certain ArUco can be treated as feature points. I defined the center of 4 ArUcos as the origin (0, 0) of the world, then we know the 3-D coordinates of 4 corner points. After we detect the ArUco in the images, we will know their coresponding 2-D coordinates in the images. So, we can solve and find the translation and rotation vectors using the equation above. Here, translation and rotation vectors are the so-called camera pose (orientation and position).

Finally, I used MoveIt! library in ROS framework to control a UR5 robot, which I installed a camera. Keeping moving the robot, its pose (same as the camera pose) will change, then I can verify my algorithm compare to the pose data in the robot control panel.

Actually, this is a review section. I have done this project a year ago, so I forget the accuracy data from my research. I just remember this algorithm worked well.

## OpenCV Function

• solvePnP
• solvePnPRansac