C# Forms - Setting the text enter cursor

I'm currently making a basic calculator form and to this point I've used a Regex method to restrict any character that's not an operator(+,-,*,etc.) or a number.

I've also implemented it so if they enter an illegal character an error msg pops up as well as deleting the last char in the text box entered.

Regex LegalChars = new Regex(@"[^0-9 + - * / % .]");

MatchCollection Matches = LegalChars.Matches(UserInput.Text);

if (Matches.Count > 0) {
    MessageBox.Show("You can only enter what is shown on the calculator\nI.e. No letters or different symbols");            
    UserInput.Text = DeleteLastChar(UserInput.Text); //couldn't get .TrimEnd to work so i made my own function         
}

But I noticed if you paste something like "5*a+6-12" it will delete every single character until it deletes the a(i.e. its now "5*"), also if you enter an illegal character in the middle of the textbox chars it does the same thing.

Any solutions to this problem? i.e. A way to delete all illegal characters but only the illegal characters. Or better yet the exact way the Windows calculator is implemented - you can't enter illegal chars at all