Most of the man-made fibres (MMF), in product types and volume, are made out of synthetic polymers; the feedstock of which is oil-based. In the first instance, this non-renewable resource is in view of sustainability not the preferred resource compared to (rapidly) renewable sources. But this has to be considered in the context of other resources like land use and water use, needed for the production of natural fibres.
On top of that, only minor volumes of the oil resources are used for man-made fibres. Oil-based feedstock used in European production of man-made fibres amounts to an estimated 0.06% of world oil production. At a global level, MMF consume about 0.8% of the crude oil production.
Actually, still some 90 % of the petroleum produced is directly used for heating and transport. It is clear that saving on oil, has to be focused on renewable alternatives for transport and heating, where huge savings can be realised.
And without man-made fibres there would be not enough land available, if they were completely substituted by natural fibres like cotton and wool. Substitution of man-made fibres by natural fibres, will not bring a huge saving on crude oil production and use, but will go at the expense of arable land and at the expense of water. Both are essential for people daily life and will be needed for the growing population and should primarily not be used for textiles.