The Minister with responsibility for Irish Water says it would set a 'dangerous precedent' to write off the water charges people have not paid.
The Government is set to introduce legislation to suspend the controversial charges for nine months while the future of Irish Water and water charges are examined.
However, only 64% of customers are reported to have paid their water bill to date - with Simon Coveney admitting there was a 'fall-off' after the general election amid speculation the utility might be abolished.
Speaking to Newstalk Breakfast, Minister Coveney said: "We've made it very clear in our Programme for Government that people who have paid are not going to be disadvantaged versus people who haven't [...] When we suspend charges, what we want is to suspend charges and penalties for the nine-month period to take the heat out of the situation."
He said that it is view that Irish Water should be "pursuing people who haven't paid. When there are charges that people are legally bound to pay, I think it would set a very dangerous precedent for us to simply write them off. That is not what I'd be advocating."
He explained that Fine Gael wants to pursue people who haven't paid in a way that will allow them to afford the charges. However, he added that "I am not going to make a fool out of people who have paid their water charges because they believed it was the right thing to do, and they believed they had a responsibility to do that."
Irish Water's parent company, Ervia, has previously said that 64% of customers had paid the charges at the end of its fourth billing cycle, which covered services for the last three months of 2015.
According to Ervia, 975,000 customers had paid "all or part of their bills" by the end of March.
However, company revenue fell dramatically as debate over charges intensified around the time of the general election.
Money taken in from the fourth billing cycle added up to €33.4 million, compared to €42.3 million for the third, €38 million for the second and €30.5 million for the first.
Earlier this month, Taoiseach Enda Kenny's latest comments on water charges were described a 'slap in the face' to voters by Opposition politicians.