AlmostSurely

Work backwards from the last roll. Let $V_k$ be the expected payoff under optimal play when $k$ rolls remain, including the one about to be made. A rational player keeps a value $x$ exactly when $x$ beats the continuation value. Hence

V_k=\mathbb{E}\bigl[\max(X,V_{k-1})\bigr],\qquad X\sim\mathrm{Unif}\{1,\dots,6\}.

One roll left. You must accept the result, so

V_1=\frac{1+2+\dots+6}{6}=\frac{7}{2}.

Two rolls left. Keep the second roll if it exceeds $\tfrac72$ , that is, if it is $4$ , $5$ or $6$ . Otherwise reroll. Then

V_2=\frac{4+5+6}{6}+\frac{3}{6}\cdot\frac{7}{2}=\frac{15}{6}+\frac{7}{4}=\frac{17}{4}.

Three rolls left. Keep the first roll if it exceeds $\tfrac{17}{4}$ , that is, if it is $5$ or $6$ . Otherwise reroll. Then

V_3=\frac{5+6}{6}+\frac{4}{6}\cdot\frac{17}{4}=\frac{11}{6}+\frac{17}{6}=\frac{14}{3}.

The optimal strategy is therefore:

keep the first roll only if it is $5$ or $6$ ;
keep the second roll if it is $4$ or more;
accept the third roll whatever it is.

The fair price equals the expected payoff of this strategy:

\boxed{\frac{14}{3}\approx 4.67}

Key idea. Each extra roll raises the continuation value ( $3.5$ , then $4.25$ ), and that value sets the keep threshold one stage earlier.