DTLS Protocol Weakness Analysis And Solutions

As the rapid development and popularity of Internet applications,the security protocols of application layer have become an important means to protect the communication privacy.Since the advent of the Transport Layer Security(TLS)protocol which protect TCP traffic,the Datagram Transport Layer Security(DTLS)protocol for protecting UDP communications has also emerged.DTLS 1.0 and 1.2 are based on TLS 1.1 and 1.2 respectively,so most of the specifications of DTLS are the same as TLS.One of their differences is that an explicit sequence number is used in the record layer part of a DTLS packet.The sequence number increases from 0 and is applied for anti-replay attack.However,the specifications of the anti-replay mechanism of DTLS is flawed and can be used by attackers to implement DoS attack against DTLS servers.Starting from the DTLS specifications,this thesis analyzes the similarities and differences of anti-replay and anti-DoS parts between DTLS and TLS to reveal the weakness of that anti-replay part of DTLS.Two different solutions are also designed in this thesis.The main research contents and innovations of thesis thesis are as follows:1.A weakness of the anti-replay part of DTLS was found.This thesis describes the detail of this defect and demonstrates it through several attack experiments.Rather than disconnect the DTLS connection when the server is under attack,the DTLS specification tolerates that a receiver can maintain the DTLS connection when it detects replayed sequence number within the received message,which causes a security issue.2.A solution based on modifying the protocol specification is designed.This solution add a detection mechanism on the basis of the original DTLS specifications.The mechanism stipulates for that a DTLS server should disconnect the DTLS communication when receiving the replayed sequence number continuously up to a upper limit times,which will protect the server from the attackers.3.A solution based on monitoring time is designed.By monitoring the length of time of DTLS receiving operations in DTLS handshake and formal communication,whether the server is under the DoS attack based on the replayed sequence number can be determined.The DTLS server records the start time of every receiving operation,then it does subtraction with the current time in the time monitor.The obtaineddifference is the judgment basis to determine whether the server is under the attack.This solution is more portable than the means to modify the protocol specifications.