To the extend it is needed, here are my 2 cents about how I understand those Drupal thingies:
- an entity (introduced as of D7) is something to represent a user, a node, a group, a message, etc. If you're familiar with a relational DBMS, it's the equivalent of a table.
- an field is typically some sort of "attribute" to provide more details about an entity, something to represent a name, a location, a date, a color, the key of a related entity, etc. If you're familiar with a relational DBMS, it's the equivalent of a column of a table.
To continue the comparison with a relational DBMS: saying that "field" is a synonym of an "entity", is like saying that a "column" (of a table) is a synonym of a table. That's for sure not correct (actually completely wrong).
Or to add a Drupal example, if you're familiar with the Rules module, how about Rules Conditions like "Entity has Field" or "Entity is of type". Or Rules Actions like "Fetch Entity" (not something like "Fetch Field").
- a field formatter has to do with how the value of some field is to be formatted (to be displayed), that's not what a field is about. In Drupal Commerce, prices are stored in fields with values that correspond to cents, however when they are shown (in a cart, etc) they are formatted as EURs, or USDs, etc, with a separator (comma or point) in front of the last 2 digits.
- an entity reference is about a link (a pointer) to some entity, something which is actually the key that helps retrieving the entity that the entity reference is pointing to. That's not the same as an entity itself. In a relational DBMS, this is what a "foreign key" is about.
Update: some related searches: