Testing equality of two Ristretto points means testing whether they
are equal in the quotient group, i.e., whether they lie in the same
coset of E[4] (for the cofactor-8 case) or
E[2] (for the cofactor-4 case).
Equality testing of points on the Edwards curve requires comparing to
affine coordinates, which requires an expensive inversion. However,
testing whether two points lie in the same coset can be done in
projective coordinates, making it actually easier than equality
testing in the original non-quotient group.
The Decaf paper proves that in the cofactor-4 case, to test equality of
P1=(X1:Y1:Z1:T1)
and
P2=(X2:Y2:Z2:T2)
it's sufficient to test whether
X1Y2=Y1X2.
In the cofactor-8 case, we need to test equality not modulo
E[2] but modulo E[4]. This means we
need to check whether either P1≡P2(modE[2]) or P1+Q4≡P2(modE[2]), where
Q4 is a 4-torsion point.
Write P1+Q4=(X1′:Y1′:Z1′:T1′).
In affine coordinates, P+Q4=(y/a,−xa),
so that X1′=Y1/a, Y1′=−X1a.
The equation
X1′Y2=Y1′X2
then becomes
Y1Y2/a=−aX1X2
or equivalently
Y1Y2=−aX1X2.
So, to check equality modulo E[4], it's sufficient to check whether
X1Y2=Y1X2orY1Y2=−aX1X2.