I think that works fairly well.
Or perhaps just say it adds a Setback for each hindering character.
But the problem here is that something like Stealth and something like Mechanics are very different. With Mechanics, someone who doesn’t know what they’re doing is a nuisance, but it doesn’t make you incapable of accomplishing the goal. If you have a Weequay who’s as dumb as a box of rocks assisting a mechanic with Int 4 and 2 Mechanics, then you’re looking at a pool of YGS before you even add the difficulty.
In most cases, adding a Setback would be sufficient.
Stealth and maybe Athletics/Coordination just work differently and need to be handled differently.
Yes I think that would be something a GM could decide on, whether there’s a full affect or just the single Setback.