I understand the trauma through which you must be going regarding
this relationship. If your father has broken all the limits in your
childhood and is persisting with some wrong attitude, you are not required
to have any sort of relationship with him -- although if you do so, you
shall be greatly rewarded.
Blood relationships hold great sanctity in Islam. But if a person violates
the sanctity of such a relationship, the affected person is not required to
maintain this relationship. Consequently, if you sever your relationship
with him or keep it very limited, then this can in no way be objected to.