SetDirectory["~/KappaLib"];
<<kappaLib-1.2.m
<<helper.m

Define Metaclass I with parameters:

     alpha_i in R,    beta_i in R\0,    and    beta_i all have same sign.

kappa=emMatrixToKappa[(({{a1, 0, 0, -b1, 0, 0}, {0, a2, 0, 0, -b2, 0}, {0, 0, a3, 0, 0, -b3}, {b1, 0, 0, a1, 0, 0}, {0, b2, 0, 0, a2, 0}, {0, 0, b3, 0, 0, a3}}))];

Parameters can be permutated by a coordinate change

L12=(({{1, 0, 0, 0}, {0, 0, -1, 0}, {0, 1, 0, 0}, {0, 0, 0, 1}}));
L23=(({{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 0, -1}, {0, 0, 1, 0}}));

(* Exchange 1 <-> 2 *)
Sign[Det[L12]]emKappaToMatrix[emCoordinateChange[kappa,L12]]//MatrixForm

(* Exchange 2 <-> 3 *)
Sign[Det[L23]]emKappaToMatrix[emCoordinateChange[kappa,L23]]//MatrixForm

Note: in KappaLib, emCoordinateChange computes coordinate change for 
an (untwisted) tensor. Since kappa is a twisted tensor, we manually need to 
add the extra Sign(Det(..)) in the transformation rule.

Write out algebraic equations that kappa satisfies and eliminate variables for A and B

eta=kappa + mu  emIdentityKappa[];
LHS=emCompose[eta,eta];
AA=emMatrix["A",4,Structure->"AntiSymmetric"];
BB=emMatrix["B",4,Structure->"AntiSymmetric"];
RHS=-lambda emIdentityKappa[] + emBiProduct[rho, AA,BB]+emBiProduct[rho, BB,AA];

Since rho, A,B are all non-zero, we may scale A and assume that rho = 1

rho=1;

Definition of decomposable medium (with

eqs=simp[Union[Flatten[LHS-RHS]]];
show[eqs]

elimVars=Variables[AA.BB]

condVars=Join[Variables[kappa],{lambda,mu}]

Eliminate variables using a Gröbner basis

gb=GroebnerBasis[eqs, condVars,elimVars];//Timing
gb=simp[gb];//Timing
Length[gb]

show[Take[gb,10]]

Case 1: b1 = b2 = b3.

subs={b3->b2,b2->b1};

show[Take[simp[gb//.subs],10]]

Equation (1) implies that lambda = b1^2

subs=Append[subs,lambda->b1^2]

show[simp[gb//.subs]]

The sought equations analysed in the paper are equations (10) -- (15).

Case 2: Exactly two of b1, b2, b3 coincide

By a change of coordinates, may assume that b2 = b3, b1 != b2

subs={b3->b2};

show[Take[simp[gb//.subs],3]]

By equation (3) we have lambda = b1^2 or lambda = b2^2

Case 2: Subcase 1: lambda = b1^2

Case = Append[subs,lambda->b1^2]

tmp=simp[gb//.Case];
show[Take[tmp,3]]

Since b1 != b2, equation (3) implies that mu = -a1

Case = Append[Case,mu->-a1]
tt=simp[gb//.Case];
show[tt]

Equation (2) implies that a1=a3
Equation (3) implies that a1=a2

Case = Append[Case,a3->a1];
Case = Append[Case,a2->a1];

show[simp[gb//.Case]]

Equation (2) implies that b1 = b2. Thus Case 2: Subcase 1 leads to 
a contradiction

Case 2: Subcase 2: lambda = b2^2

Case = Append[subs,lambda->b2^2]

show[simp[gb//.Case]]

Since b1, b2 have same sign and b1 != b2, equations (20) and (21) imply that mu=-a2=-a3

Case = Append[Case,mu->-a2];
Case = Append[Case,a3->a2];

tt=simp[gb//.Case];
show[tt]

We have found a solution.

emKappaToMatrix[kappa//.Case]//MatrixForm

For this medium we know that the Fresnel surface is a double light cone.

Case 3: All of b1, b2, b3 are distinct

subs={};
show[Take[simp[gb],8]]

Equation (8) and (5)--(7) imply that lambda = bi^2 and mu = -ai for some i=1,2,3

Case 3: Subcase 1: i=1

subs={lambda->b1^2, mu->-a1};
show[simp[gb//.subs]]

Since b1 != b3, equation (2) implies that a1 = a2

subs=Append[subs,a2->a1];
show[simp[gb//.subs]]

Since b1!= b2 and b1, b2 have same sign, equation (2) implies that a3=a1

subs=Append[subs,a3->a1];
show[simp[tmp//.subs]]

Equation (2) implies that b1=b2. This contradicts that beta_i are all distinct (and of same sign). Hence Case 3: Subcase 1 leads to a contradiction.

Note: By symmetry, the case i=1 should be enough. However, since cases i=2,3 require some manual simplification (motivated by the i=1 case), these cases are provided below for completeness.

Case 3: Subcase 2: i=2

subs={lambda->b2^2, mu->-a2};
show[simp[gb//.subs]]

The factor in equation (2) can be rewritten

fa= (((((((a2-a3))^2)+(b2^2)))^2)+2 ((((a2-a3))^2)-(b2^2)) (b3^2)+(b3^4));
fa2=((((a2-a3))^2)+(((b2-b3))^2)) ((((a2-a3))^2)+(((b2+b3))^2));
Simplify[fa-fa2]

Thus, since b2 != b3, equation (2) implies that a1 = a2

subs=Append[subs,a2->a1];
show[simp[gb//.subs]]

Since b1!= b2 and b1, b2 have same sign, equation (2) implies that a3=a1

subs=Append[subs,a3->a1];
show[simp[tmp//.subs]]

Equation (2) implies that b1=b2. This contradicts that beta_i are all distinct (and of same sign). Hence Case 3: Subcase 2 leads to a contradiction.

Case 3: Subcase 3: i=3

subs={lambda->b3^2, mu->-a3};
show[simp[gb//.subs]]

The factor in equation (2) can be rewritten

fa= (((((((a1-a3))^2)+(b1^2)))^2)+2 ((((a1-a3))^2)-(b1^2)) (b3^2)+(b3^4));
fa2=((((a1-a3))^2)+(((b1-b3))^2)) ((((a1-a3))^2)+(((b1+b3))^2));
Simplify[fa-fa2]

Thus, since b1 != b3, equation (2) implies that a2 = a3

subs=Append[subs,a3->a2];
show[simp[gb//.subs]]

Since b3!= b2 and b3, b2 have same sign, equation (2) implies that a2=a1

subs=Append[subs,a2->a1];
show[simp[tmp//.subs]]

Equation (2) implies that b1=b2. This contradicts that beta_i are all distinct (and of same sign). Hence Case 3: Subcase 3 leads to a contradiction.