The Association of Garda Sergeants and Inspectors (AGSI) is warning against the hasty reassignment of officers.
It comes after a Garda Inspectorate report published yesterday made 81 recommendations to improve the force.
The report was also highly critical of structures within An Garda Siochána - saying that in some areas we are 30 years behind the UK for best policing practices.
It said there is a critical of the lack of community gardaí, who are primarily Dublin based.
There are just 540 community gardaí out of a total force of 12,800 - with none in either Mayo or Kildare.
The lack of a cyber crime unit within the force was also raised - with the inspectorate revealing that there is a four year backlog in the analysis of computers involved in some cases.
It also recommends a streamlining of management structures within the force, which could free up 250 gardai to hit the beat in a matter of days.
Head of the Garda Inspectorate, Bob Olson, says that could be done in a matter of days by streamlining management structures and reassigning personnel.
And John Jacob, assistant general-secretary of the AGSI, says he accepts that may be the case - but is warning against a hasty reaction.
"If Bob says there's a lot of people that can be released, I wouldn't disagree with him" he told Newstalk Breakfast.
"I'd like to look at how you can backfill those people if you took them out straight away - because it's not a matter of just walking down the street and picking up people and putting them into administrative positions presuming the job is going to get done effectively".
"I wouldn't disagree that there could be a reassignment of personnel, but I would cautions about racing to reassign people without having somebody to do the job" he added.
While Justice Minister Frances Fitzgerald has denied there is 'a crisis' in the force.
But she told the Pat Kenny Show here on Newstalk there is clear scope for more civilianisation of the gardaí.