Spasticity is a common complication for patients with stroke, but only few studies investigate the relation between spasticity and voluntary movement. This study proposed a novel automatic system for assessing the severity of spasticity (SS) of four upper-limb joints, including the elbow, wrist, thumb, and fingers, through voluntary movements. A wearable system which combined 19 inertial measurement units and a pressure ball was proposed to collect the kinematic and force information when the participants perform four tasks, namely cone stacking (CS), fast flexion and extension (FFE), slow ball squeezing (SBS), and fast ball squeezing (FBS). Several time and frequency domain features were extracted from the collected data, and two feature selection approaches based on recursive feature elimination were adopted to select the most influential features. The selected features were input into five machine learning techniques for assessing the SS for each joint. The results indicated that using CS task to assess the SS of elbow and fingers and using FBS task to assess the SS of thumb and wrist can reach the highest weighted-average F1-score. Furthermore, the study also concluded that FBS is the optimal task for assessing all the four upper-limb joints. The overall result shown that the proposed automatic system can assess four upper-limb joints through voluntary movements accurately, which is a breakthrough of finding the relation between spasticity and voluntary movement.