Giant caloric effects were reported in elasto-, electro- and magnetocaloric materials near phase transformations. Commonly, their entropy change is indirectly evaluated by a Maxwell relation. We report the fundamental failure of this approach. We analyze exemplarily the Ni-Mn-Ga magnetic shape memory alloy. An applied field results in magnetically induced reorientation of martensitic variants, which form during the phase transformation. This results in a spurious magnetocaloric effect, which only disappears when repeating the measurement a second time. This failure is universal as the vector character of the applied field is not considered in the common scalar evaluation of a Maxwell relation.