One thing I have to say for the fight between Goku and Captain Ginyu was that Jeice was seen as an onlooker. Goku didn't expect him to join in at any point, thus he was focusing only on Ginyu. Jeice's interference isn't really what I'd compare a hypothetical fight with Freeza in Vegeta's mind to.
Krillin and Gohan were dropping shit bricks at Freeza's suppressed power, but that was before Vegeta gave them a pep talk. They probably weren't thinking straight when they were shaking in their boots previously.
Chapter: 294 (DBZ 100), P13.3-4
Context: as Freeza powers up
Gohan: “Th…thi…this can’t be…”
Kuririn: “To-to think that he was this incredible…”
Chapter: 295 (DBZ 101), P8.4
Context: still as Freeza powers up
Kuririn: “Wh-what ki…Th-there’s no way we can win…”
Freeza: “Naturally…Did you really think that 3 ants could win against a dinosaur?”
Chapter: 295 (DBZ 101), P9.1-5
Vegeta: “We can win! If the 3 of us fight together, we’ll be able to win somehow or another! [ ] It seems that even Freeza hasn’t noticed…These two’s battle power is steadily rising…The squirt in particular hides so much inner strength that he doesn’t even know it himself…And I’m in the process of becoming what you’re afraid of: a Super Saiyan…!”
Chapter: 295 (DBZ 101), P13.5-6, P14.1
Context: after Freeza and Vegeta grapple for awhile, after Vegeta said they could win
Freeza: “I see…Seems that wasn’t a complete lie…”
Gohan: “It-it’s true…! If we 3 fight together, we might really be able to manage something…H-he really is absolutely incredible, but we’ve gotten better too…!”
Kuririn: “I-I see…! Since coming here Vegeta’s acquired such strength that he seems to have even further surpassed his limits…B-but…F-Freeza is oddly calm…Why?...”
Looking at these statements again under closer scrutiny, it sounds more like a lot of hopefulness and desperation than firm declarations. Vegeta mentions the three of them being able to win "somehow or another," Gohan and Krillin's powers still rising, Gohan's inner power, and the idea that he's becoming a Super Saiyan. Even Gohan isn't completely confident, saying they "might really be able to manage something," rather than something more concrete. The only thing holding this all down for me is Freeza saying Vegeta's hope "wasn't a complete lie," very similar to how Cell admitted Goku's claims about Gohan weren't complete lies. With that in mind, I think the three of them should all be reasonably close to Freeza, but not enough to frighten him into thinking he could actually lose before transforming.
The only problems I see are Vegeta and Freeza's suppressed powers, which made Gohan, Krillin, and pre-merger Piccolo require new underwear, and the fact that, yeah, a hypothetical 2nd Form Freeza that's only even 20% stronger than his first form would stomp anyone that's not at least 500-550,000 in power level (which is more than anyone can legitimately put Vegeta). The latter issue is where I could see the hopefulness coming into play, but I can't quite figure out the first part.