People's creativeness being shut down? Ha! That amuses me. Whether their creativity gets shut down or not isn't an issue in my opinion. However I'll reach a middle ground here. I have a few different sets of rules in mind. If any player wants to become "creative" and make their own technique then I think they should follow these guidelines.
1. Give the technique a name of course.
2. List the hand seals by order.
3. Tell the rank of this technique (Everyone knows the ranks)
4. Tell what class this technique falls under. (Offfensive, defensive.)
5. What Classification does this technique fall under? Ninjutsu, Genjutsu, Fuinjutsu etc.
6. The range of the jutsu. (Close Range, Mid Range, Long Range)
7. And of course a description of how the jutsu is used.
8. When the steps above are completed, send it to a judge or a moderator (I doubt a moderator would have to take a look, I think a neutral judge would suffice) for approval. Someone who is completely neutral to see if the technique is not too over powered.
(And this is not my only idea.)
As someone who uses many custom techniques and rarely gets into terrible discussion with those i fight when I use them, I will put it this way;
a custom technique generally is created, and then tried out in some fashion. If disagreements come up regarding it, then the creator generally make it more clear or adjusts the technique accordingly. If none come, then the technique is "acceptable" and does not require changes until either the creator feels such or an issue arises.
To an extent, that is what spars are very useful for. Trying out new techniques and fleshing them out. Their weaknesses, their strengths, and more importantly, areas where they need improvement.
The only issue I get with this is when people adjust their custom technique to the point in which the one or two weaknesses it did have are completely resolved because these weaknesse were exploited. For example, I create flying shadow, but the one weakness that it has (other than requiring a handsign) is that it can only be used to transort a single user (in order to further distance it from a similar technique, Hiraishin).
If, after finally using it in battle, I decide after the fight to change it so that I can transport others without additional steps, then that kind of thing is what I have an issue with with custom techniques. Otherwise, it is trial, see if it gets by, and then error in application. It helps for the custom technique to have a sort of creation log to it, but that is not required.
Regarding handseals, rarely do RPers specify the precise handseals that they are using, so including those in a custom technique is largely redundant and unnecessary, in my opinion at least. To be truthful, I have seen fewer arguments regarding techniques themselves and more on how they are actually used in a fight.
In short, as a custom technique whore who has relatively few issues with people accepting his custom technique, I think that putting restrictions on custom techniquess is uneeded. Additionally, making more rules regarding canon techniques is very much not necessary most of the time (obviously hiraishin and Edo Tensei are two examples that come to mind right now) and would only serve to make RP more regulatory madness.
Imho, maybe rules specifically on special S-ranked techniques, but other than that, the rest should be left alone.