Speaking Villagers - ChatGPT and TTS

Transform ordinary Minecraft villagers into intelligent AI-powered conversational partners!

Now with Mistral API support! Free to use with your own API key 🎉 Added player speech recognition!

Video is somewhat outdated - the mod works better now than shown in the clip.

Overview

Speaking Villagers fundamentally transforms villager interactions in Minecraft by adding AI-based conversations, dynamic friendship systems, and quest systems. Villagers now recognize their surroundings, respond to your actions, and remember previous conversations.

Each villager is assigned one of over 8,000 unique personalities, making every interaction truly special.

Some features (voice synthesis and speech recognition) work only on Windows computers.

🆕 Version 0.6 – Major Update!

• Player Voice Input: Communicate with villagers directly through microphone using speech recognition (setup described below)
• 23 Unique Voices: English, German, and multilingual voices selectable via ModMenu
• Improved Quest System: 176 different item collection tasks and 210 profession-based rewards with enhanced balance
• Expanded Personality System: Over 8,000 unique characteristic combinations for villagers
• Enhanced Gift System: Villagers now drop gifted items upon death and only pick up items dropped by players
• Experimental Language Settings: Automatic player language detection or setting preferred language
• Redesigned ModMenu Interface: Numerous new options for managing voice synthesis, memory, speech models, and other parameters
• No More Environment Variables Required – API keys can now be set via ModMenu
• Critical Bug Fixes: Resolved /speak command crashes when API keys are missing and improved overall stability

Key Features

✅ AI-Powered Smart Conversations – Talks with villagers featuring contextual responses ✅ Environmental Awareness – Villagers respond to:

• Your name, armor, held items, hotbar, health and hunger
• Biome, weather, and time of day
• Nearby mobs, blocks, and structures
• Active potion effects and game mode
• Reputation and friendship level
• Being indoors, in bed, swimming, or riding

✅ Friendship System – Befriend villagers and they'll follow you ✅ Quest System – Villagers offer procedurally generated tasks ✅ Gift System – Villagers pick up items and comment on them (activated via ModMenu) ✅ 8000+ Different Personalities – Every villager is unique ✅ Windows Voice Support – Villagers speak aloud using Windows Speech API ✅ Compatibility with Talking Mobs mod

🛠️ Full Customization via ModMenu: AI model selection, limit settings, system toggling, and much more!

API Setup (Mistral or OpenAI)

🎉 Free Option – Mistral

• Create a free account on mistral.ai
• Generate API key – can be found in the "La plateforme" section after logging in
• Add to system as environment variable: MISTRAL_API_KEY=your_key
• From version 0.6, it's not mandatory to set API keys via environment variables – now configurable through ModMenu!

Mistral usage limit – 1,000,000,000 tokens per month, impossible to exceed when using only this mod

💰 Paid Option – OpenAI

• Requires OpenAI account and API key
• Add to system as: OPENAI_API_KEY=your_key
• From version 0.6, it's not mandatory to set API keys via environment variables – now configurable through ModMenu!
• Billed per token used

No API key = No AI responses = No charges. Choose model via ModMenu.

After setting environment variables, restart your computer. Or configure via ModMenu (no restart required)

Installation

1. 🧩 Mod Installation Place the .jar file in the mods folder.

2. 🔊 Install Natural Voice SAPI Adapter for improved villager speech (Windows only) Without this, voices will sound robotic.

3. 🔐 API Key Setup Choose OpenAI or/and Mistral – both supported. From version 0.6, it's not mandatory to set API keys via environment variables – now configurable through ModMenu!

4. 🎤 Speech Recognition Setup (optional)

   • Run the mod once and enable speech recognition in ModMenu
   • When attempting to speak, the mod will display the required model path (usually ./AppData/Local/SpeakingVillagers/vosk_model)
   • Download a Vosk model and extract contents into the specified folder
   • Restart the game and try talking to a villager

Larger models require powerful hardware.

Commands

• /speak <message> – Say something to the nearest villager
• /befriend – Manually start friendship with nearest villager
• /unfriend – Manually end friendship with nearest villager
• /quests – Show active quests
• /villagertraits – Show personality traits of all nearby villagers
• /tokens – Show number of tokens used (mainly for debugging, doesn't affect gameplay)

Compatibility

✅ Compatible with most other mods ⚠ Some features require Windows (uses Windows Speech API for voice synthesis, Vosk for speech recognition) ⚠ Tested only in singleplayer – should work on servers when installed on both client and server, but full multiplayer optimization still in development ⚠ Java 21 required. Minecraft 1.20.x users must manually specify Java Executable in launcher settings to Java 21 (javaw.exe), as it defaults to Java 17 which may cause launch crashes

Planned Features for Future Updates

• Full multiplayer compatibility
• Support for other TTS APIs like ElevenLabs – will enable voice synthesis on other operating systems, more voices and languages
• Interface and visual improvements
• Translation of settings into more languages