Difference between "denied" and "refused"

[Rida*]

hi sir,
what is the difference between "denied" and "refused".
is it right,
he refused having stolen my pen.
thank you.

[Everafter]

He denied having stolen my pen.

[Everafter]

Deny: to say that something someone has said about you isn't true and the opposite is " admit".

Refuse: to say firmly that you are not willing to accept or do something and the opposite is " accept".

[Yuppie]

I personally don't think that denying is limited to not accepting that something which is said about someone is not true, unless if this statement is wrong:

" Mary was denied access to her sister's house"