Make the yolk mixture: In a large heatproof bowl, whisk egg yolks with granulated sugar until combined. Whisk in lemon juice, lime juice, lemon zest, lime zest, and salt until smooth.
Set up a double boiler: Add about 5 cm (1–2 inches) of water to a saucepan and bring to a boil. Reduce to a steady simmer (water should not touch the bowl).
Cook the custard: Place the bowl over the simmering water and whisk continuously until the mixture thickens, 5–10 minutes, resembling a hot, pourable custard. Remove from heat.
Cool completely: Set the bowl over an ice bath and let cool completely, stirring occasionally, 15–20 minutes.
Whip the cream: In a separate bowl, beat whipping cream to stiff peaks.
Fold together: Gently fold the whipped cream into the cooled custard until evenly combined and airy.
Freeze: Line a loaf pan with plastic wrap, leaving overhang. Pour mixture into the pan, smooth the top, cover with plastic wrap, and freeze overnight.
Make topping: Beat whipping cream to stiff peaks. Add sugar and vanilla and beat until smooth and stable.
Unmold: Lift semifreddo out using plastic wrap and invert onto a serving plate.
Top + set: Spread or pipe whipped cream topping over the semifreddo. Freeze 30–45 minutes to firm up.
Serve: Let sit 5 minutes at room temperature, then slice into 2 cm (1-inch) pieces.