With the widespread application of RFID technology in military, financial, public security and other fields, people put forward higher requirements for the security performance of RFID system. This paper designs a RFID system mutual authentication protocol based on the elliptic curve cryptography (ECC) by using ECC characteristics of short key, high security and small storage space, and analyzes and proves formally that the protocol can achieve the desired security targets by using BAN logic. Compared with other similar application protocols, the protocol has good reliability in resisting tracking attack, denial of service attack, retransmission attack and impersonation attack, and can satisfy mutual authentication, confidentiality, anonymity and forward security. The protocol requires for 3 times elliptic curve scalar multiplication operations, which reduces the tag computational cost by 60% compared with other protocols, while improves the implementation efficiency of the RFID system by 70%. The protocol applies to the key areas such as military logistics management and confidential documents protection.