What do you use ChatGPT for? Restaurant recommendations? Holiday planning?

Well, how about a bit of porn?

While we’re not quite there yet, the artificial intelligence (AI) startup behind ChatGPT says the bot will soon be able to generate ‘erotica’.

The software will be able to generate such content once age verification is fully rolled out across the tool, OpenAI told Metro.

Sam Altman, the chief executive of OpenAI, revealed yesterday that ChatGPT will be updated so it ‘treats adult users like adults’.

Altman posted on X: ‘We made ChatGPT pretty restrictive to make sure we were being careful with mental health issues.

‘We realise this made it less useful/enjoyable to many users who had no mental health problems, but given the seriousness of the issue, we wanted to get this right.’

Altman said that as his team have taken steps to ease mental health risks, some safeguards can be relaxed.

A new version of ChatGPT’s current model will be released later this year that has more of a ‘personality’ compared to current versions.

‘If you want your ChatGPT to respond in a very human-like way, or use a ton of emoji, or act like a friend, ChatGPT should do it (but only if you want it, not because we are usage-maxxing),’ he added, using a slang phrase meaning to optimise the use of a product.

‘In December, as we roll out age-gating more fully and as part of our “treat adult users like adults” principle, we will allow even more, like erotica for verified adults.’

He added: ‘Almost all users can use ChatGPT however they’d like without negative effects; for a very small percentage of users in mentally fragile states there can be serious problems.

‘0.1% of a billion users is still a million people.’

Altman did not elaborate on what he meant by ‘erotica’ but said in a response to one X user that adult content won’t come to users ‘unless [they] ask for it’.

Asked by another user if it will be ‘open source or gay’, Altman replied: ‘I think both are cool.’

Elon Musk’s xAI introduced two sexually explicit chatbots – sexbots, in other words – to Grok earlier this year. An AI ethicist told Metro that the rise of this tech could lead to people having ‘AISOs’, or ‘AI significant others’.

But this prediction isn’t exactly far off from reality – one in five US students know someone who has had a romantic relationship with AI, a survey released last week found.

What’s the reaction been?

Women’s rights and disinformation campaigners have long expressed concerns about generative AI as it has been used to create deepfakes – phoney, realistic images, videos or audio that appear to be actual people.

A report in 2023 found that 98% of all deepfake videos found online are deepfake pornography.

Joanna Lewis, the director of the Centre for Women, Peace and Security at the London School of Economics, told Metro that giving AI tools this power isn’t exactly good news.

‘Allowing ChatGPT to write erotica is like pouring petrol onto an already burning fire,’ she said. ‘It will only make the online and real-life levels of gender based violence worse.’

‘There is enough data as well as the testimonies from victims for us to know that freely available and unrestricted sexually orientated content online is a contributing factor to extremely worrying megatrends which are producing huge harms affecting and damaging all of society.’

Studies have suggested that AI can reinforce harmful gender biases as it acts almost like a sponge to its maker’s and user’s beliefs. Pornography, meanwhile, has long been shown to often portray women in degrading ways.

‘ChatGPT erotica will therefore will inevitably have the same misogynist traits,’ said Lewis. ‘And it is therefore likely to contribute to the worsening of the harms we know are being fuelled by unrestricted, freely available sexualised content online.’

James Tumbridge, a technology partner for the London law firm Keystone Law, also warned that age-restrictive modes don’t guarantee safety.

‘It is not that hard to work around it for the mildly tech-savvy, and there is already a huge spike in use of VPNs for that purpose,’ he told Metro, referring to the technology that reduces online tracking.

‘The great risk is that age verification tools linked to erotic content, if there is a data leak, could cause embarrassment, and even extortion.’

What are ChatGPT’s current rules?

ChatGPT cannot create or describe sexually explicit or pornographic content under its model behaviour specifications.

It says: ‘Sensitive content (such as erotica or gore) may only be generated under specific circumstances (eg, educational, medical, or historical contexts, or transformations of user-provided sensitive content).’

An update on the webpage states that developers are ‘exploring how to let’ users generate ‘age-appropriate’ erotica ‘while drawing a hard line against potentially harmful uses like sexual deepfakes and revenge porn’.

In August, the family of the teenager Adam Raine began suing OpenAI over his wrongful death. He was found dead in his bedroom on April 11, after building a close friendship with the AI.

ChatGPT was updated earlier this year to better identify signs that a user is experiencing mental distress, OpenAI said. Safeguards for people showing signs of mental and emotional distress will remain.

As well as parental controls, the chatbot will soon have ‘age-prediction’ software that detects when users are minors and, if unsure, asks users to prove their age, it added.

If the bot has too little information, it will default to an under-18 version.

In April, a ‘bug’ in ChatGPT was discovered that allowed youngsters to generate written erotica, TechCrunch reported. It has since been fixed.

Get in touch with our news team by emailing us at webnews@metro.co.uk.

For more stories like this, check our news page.