Validated optional nested form in Symfony2

Is there a way in Symfony2 to add an optional nested form while using cascade validation? In other words, say I have a user form with a nested address form (->add('adresse', new AddressType(), array('required' => false))). This address is not required BUT must be validated in case the user specify it ('cascade_validation' => true).

UPDATE

Relevant entities and forms :

User entity (Getters & Setters are classical, Symfony generated):

class User
{
    [...]

    /**
     * @var \Address
     *
     * @ORM\OneToOne(targetEntity="Address", cascade="persist")
     * @ORM\JoinColumn(
     *        name="address_id", referencedColumnName="id"
     * )
     */
    private $address;

    [...]
}

The address entity is classical, no bidirectionnal relation to user.

User form

    class UserType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            [...]
            ->add('address', new AddressType(), array('required' => false))
            [...]
        ;
    }

    public function setDefaultOptions(OptionsResolverInterface $resolver)
    {
        $resolver->setDefaults(array(
            'data_class' => 'Xentia\FuneoBundle\Entity\User',
            'cascade_validation' => true
        ));
    }

    public function getName()
    {
        return 'user';
    }
}

The address nested form is classical

As you can see, the is a quite classical and straightforward code. The only particular case is that address is optional. Leading to an validation error only in the case that the address was previously set (and, thus, exist in the DB and as a not null relation with the user) and the user want to unset it (all address fields are left empty). It seems that if the related address has not an actual instance it can still be optional. But, if an instance of the address exist and is linked with the user, it can not be optional anymore.

So, what should it do: When the user update his profile, if there was a previously set address and he try to unset it, user->address should be set to NULL and the address should be deleted. What does actually happens: The form throw a validation error saying that the address fields are not set. But, as the address is optional, such validation error should not occur. And, in fact, it does not occur if the address was not set before the user update his profile, it only occurs if there was an address previously set.