-brown>){kind=icon}
An app to download novels from online sources and generate e-books.
Discord: https://discord.gg/wMECG2Q
Telegram: https://t.me/epub_smelter_bot
This application uses Calibre to convert ebooks.
Install it from https://calibre-ebook.com/download
Without it, you will only get output in epub, text, and web formats.
⏬ Windows: lightnovel-crawler v2.22.0 ~ 23MB
In Windows 8, 10 or later versions, it might say that
lncrawl.exeis not safe to dowload or execute. You should bypass/ignore this security check to execute this program. Actually, I am too lazy to add proper configuration files to solve this issue. Excuse me please 😇.
To get older versions visit the Releases page
⏬ Linux: lightnovel-crawler v2.19.4 ~ 24MB
Copy it to
/usr/binor~/.local/binto make it accessible in terminal. Or, you can create a launcher to easily access the app.
📦 A python package named lightnovel-crawler is available at pypi.
Make sure you have installed Python v3.5 or higher and have pip enabled. Visit these links to install python with pip in Windows, Linux and MacOS. Feel free to ask on the Discord server if you are stuck.
To install this app or to update installed one via pip, just run:
$ pip install --user -U lightnovel-crawlerRemember, in some cases you have to use python3 -m pip or pip3 or python -m pip. And you do not need --user option, if you are running from root.
Next, open your terminal and enter:
$ lightnovel-crawler
# Or, a shortcut:
$ lncrawlTo view extra logs, use:
lncrawl -lll
📱 Using Termux, you can run this app in your android phones too. Follow this instructions:
apt update && apt upgradetermux-setup-storagepkg install ndk-sysroot make python zlib clangpkg install libxml2 libxslt libiconv libcrypt libffi zlib libjpeg-turbopkg install nodejs-ltspip install -U lightnovel-crawler to install the latest version of this app.cd ~/storage/downloads to store novels there.lncrawl to start.pip install -U lightnovel-crawler again to install the latest updates.Visit this link to get started with the telegram bot: https://t.me/epub_smelter_bot
Join our server: https://discord.gg/7A5Hktx
Or, visit this link to install discord bot to your own server: https://discordapp.com/oauth2/authorize?client_id=537526751170002946&permissions=51264&scope=bot
Send !help to open the bot help message.
$ git clone https://github.com/dipu-bd/lightnovel-crawler$ pip install --user -r requirements.txt$ python __main__.py
# Or, in short,
$ python .Simply fill out the environment variables and you get a running instance.
$ lncrawl -h
================================================================================
📒 Lightnovel Crawler 🍀2.22.0
https://github.com/dipu-bd/lightnovel-crawler
--------------------------------------------------------------------------------
usage: lncrawl [options...]
lightnovel-crawler [options...]
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l Set log levels. (-l = warn, -ll = info, -lll = debug).
--list-sources Display a list of available sources.
-s URL, --source URL Profile page url of the novel.
-q STR, --query STR Novel query followed by list of source sites.
-x, --sources Display the source selection menu while searching.
--login USER PASSWD User name/email address and password for login.
--format E [E ...] Define which formats to output. Default: all.
--add-source-url Add source url at the end of each chapter.
--single Put everything in a single book.
--multi Build separate books by volumes.
-o PATH, --output PATH
Path where the downloads to be stored.
--filename NAME Set the output file name
--filename-only Skip appending chapter range with file name
-f, --force Force replace any existing folder.
-i, --ignore Ignore any existing folder (do not replace).
--all Download all chapters.
--first [COUNT] Download first few chapters (default: 10).
--last [COUNT] Download last few chapters (default: 10).
--page START STOP. The start and final chapter urls.
--range FROM TO. The start and final chapter indexes.
--volumes [N [N ...]]
The list of volume numbers to download.
--chapters [URL [URL ...]]
A list of specific chapter urls.
--bot {console,telegram,discord,test}
Select a bot. Default: console.
--shard-id [SHARD_ID]
Discord bot shard id (default: 0)
--shard-count [SHARD_COUNT]
Discord bot shard counts (default: 1)
--suppress Suppress all input prompts and use defaults.
ENV [chatbots only] Pass query string at the end of all options. It will be use instead of .env
file. Sample: "BOT=discord&DISCORD_TOKEN=***&LOG_LEVEL=DEBUG"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Open your console and type lncrawl --version first to check if you have installed it properly.
Here are some example usage of the app:
To start an interactive session: lncrawl
To download using an url: lncrawl -s https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/
To search novels: lncrawl -q "Strongest Sword God"
To search novels from selected sources: lncrawl -q "Strongest Sword God" --sources
To download all chapters: lncrawl --all
To download first 25 chapters: lncrawl --first 25
To download all between two chapters: lncrawl --range 10 30
To download all between two chapter links: lncrawl https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/chapter-2205 https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/chapter-2211
To download a specific volumes: lncrawl --volumes 2 3
To define output path: lncrawl -o "D:\Lightnovels\reincarnation-of-the-strongest-sword-god"
To delete the output folder if exists: lncrawl -f
To ignore the output folder if exists: lncrawl -i
To resume download where is has been left previously: lncrawl -i
To specify output formats: lncrawl --format epub pdf mobi
To display list of supported sources: lncrawl ---list-sources
If you provide an option in the argument, it will skip it in the interactive session.
If you want to disable all interactive prompts, pass --suppress at the end.
You can stack up options like this: lncrawl -s https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/ -o "D:\Lightnovels\reincarnation-of-the-strongest-sword-god" --last 50 -i --format pdf --suppress
There are two chatbots available at this moment: Telegram and Discord. To run your own bot server, follow these instructions:
# Clone this repository
$ git clone https://github.com/dipu-bd/lightnovel-crawler
# Install requirements
$ pip3 install --user -r requirements.txt
$ pip3 install --user -r bot_requirements.txt
# Edit the environment variables
# You should give your API keys and log info here
# Also specify which bot server you want to run
$ cp .env.example .env
$ vim .env
# Run the server using:
$ python3 .There is a server.sh script to run a bot in ubuntu servers. It will basically execute the python __main__.py and send the task to run in background. I use it to run my discord bot in the server.
You are very welcome to contribute in this project. You can:
sources/_template_.py as template.README.mdtest_user_inputs variable in lncrawl/bots/test/test_inputs.pybots/_sample.py as template.bots/__init__.py file.Request new one by creating a new issue.
| Rejected Sources | Reason |
|---|---|
| http://fullnovel.live | 403 - Forbidden: Access is denied |
| http://moonbunnycafe.com | Does not follow uniform format |
| https://anythingnovel.com | Site broken |
| https://indomtl.com | Does not like to be crawled |
| https://lnindo.org | Does not like to be crawled |
| https://mtled-novels.com | Domain is expired |
| https://novelgo.id/ | Removed |
| https://www.flying-lines.com | Obfuscated content |
| https://www.jieruihao.cn | Unavailable |
| https://www.noveluniverse.com | Site is down |
| https://www.novelupdates.com | Does not host any novels |
| https://www.novelv.com | Site is down |
| https://yukinovel.id | Removed |
| https://www.rebirth.online | Site moved |
| http://gravitytales.com | Redirects to webnovel.com |
| https://novelplanet.com | Site is closed |