Open Preferences → App Passwords (or the Security tab in Classic). Each app password is listed with when it was created and when it was last used.
Revoking app passwords never changes your main password or your webmail sign-in. If you think someone knows your main password, change it as well.