Need advice - Calibre for converting and managing fanfiction

[valiha]

I need some advice on how to use Calibre to convert and manage my vast fanfiction library. I have tried using this program in the past, installing and uninstalling several times. I cannot stand the fact that it creates a copy of every file I add to it. I understand the reasons for it now though I didn't in the past, but just knowing that it leaves a mess of folders and subfolders and files on my computer drives me batty. It says it is "designed around the concept of the logical book, i.e., a single entry in your library that may correspond to actual e-book files in several formats", but I only keep multiple formats for fanfic, so that seems reduntant to me. I have tried to find answers on the Calibre forum over at MobileRead, but most of the conversations are too technical for me. I hope some of you can at least give me some starting points.

So, I've been saving fanfiction for years, in various fandoms, because when I first started reading it, I didn't have internet at home, and net cafes charged ridiculous amounts for sitting there reading. I would bring a bunch of floppies and save favorites as txt files because they would take up the least amount of space and I could cram a lot of them on a single floppy. Those fics are still on my computer, in folders according to fandom and author. I don't bother with genres, because of the popularity of AUs in my main fandom. An author may write stories set in an Old West setting, WWII, present time, SF AU etc.; they all get saved in that author's folder. I number the series, and leave stand-alones with just the titles. I also don't bother with pairings because I'm mostly a gen reader.

I now save most of my favorites as htmls or download as epubs from AO3 to "My Fandoms > Fandom > Author" folders on my computer, and copy to my e-reader. I've been thinking about converting the old fics to htmls too, but it would be a time consuming process to do it one by one (I'm talking thousands of files, which often wrap after 65 characters). I know Calibre can do a batch convert of files from one format to another, but like I said I have issues with the way it behaves. I am wondering if I can set it up so that I convert the files I need, add tags and covers, move them to the location of my choice and then have Calibre delete the files and folders in its library created in the process.

Or if this is not possible with Calibre, could you recommend an alternative program that could do a batch convert from txt to clean, bare bones html/epub, and allow me to add tags to the resulting files? I'd really appreciate any help you can give me.

Comments

[amalthia]

The template I use to save to disk is {authors} - {title} I already have folders set up by fandom so I never went into much depth on that.

If I have a series I don't really use the series column because I don't like having extra collections to weed through on my device. Instead in the title portion I use "Repairman Jack 01 - The Tomb" and then write in Paul F. Wilson in the author column. When I save {authors} - {title} I'd get "Paul F. Wilson - Repairman Jack 01 - The Tomb" It works rather well so far. I haven't really hit the multiple books.

I thought that maybe calibre reached to that folder too and was capable of deleting files from it, not just its library.

Ah, no Calibre can't go anywhere else to delete files. It's sort of stuck with the library folder you selected and named.

Re: conversions, why do I end up with three files after converting txt to epub: the actual epub, an opf file, and a jpg image (the cover)?

It's an epub standard thing. If you were to make an epub from scratch you'd have these separate files. Epub is basically a zip file that houses the components of the ebook, which include html, opf, cover, toc.ncx, mimetype, and possibly a metadata folder.

The epub is the single file but within the epub you can have hundreds of files (depending on how much artwork is embedded and how long the story is.) The opf is a mobi file hosts the location of all the html files in the epub so if you wanted to create an ebook using the mobipocket software I think it'll let you. I could be wrong the opf and what it does, but my impression is that it's sort of a table of contents of the epub html files.

Isn't the cover supposed to be inside the epub?

It should be inside the epub container. Where are you finding your covers when you save your epub to disk?

It is simply their existence that causes my automatic reaction of DO NOT WANT. I never said it was a very rational reaction.

That's kind of a shame because in terms of organizing ebooks, converting, and being able to find them. Calibre is the best I've been able to find. To me it seems rather organized because the Calibre Library is one folder. I also have over 3000 fan fiction stories and novels in one library and in the other about 4 gigs of ebooks I downloaded from various places. I couldn't find them in half the time I can using Calibre's search function and tagging system. I do from time to time look around and see if there is anything out there that's even better than Calibre but so far no dice.

I have my own irrationalities so I know how that goes.

My only real warning about deleting what you've converted is that if something happens to your converted copies you'll have to re-convert. I'm not sure how much time it takes you to do that, but I know for myself it takes a bit of time to convert fic. So I don't delete them from the library. (Unless they were so awful I never want to read the story again) The reason it takes me time is that I made custom columns for Read/Not Read, Word Count, pairing, Fandom, Challenge, Rating, Warnings, and Graphics. I also add summaries to all my conversions. So I can later in the future go back and see what I've read or didn't read. It's pretty much my personal archive of fan fiction.

But yeah you're not going to break the computer with what you're doing. :) I've read about some guys using Calibre to manage 20 gigs of ebooks. If that doesn't break the program I don't think you converting a few fic is going to do it.

[valiha]

My device uses folders, so I'm fine with the series column. I haven't yet decided how I'm going to transfer the converted files and what template to use (perhaps a custom column for fandom, then {authors} - {title} for stand alones, and the above template for series).

The epub is the single file but within the epub you can have hundreds of files (depending on how much artwork is embedded and how long the story is.)
I discovered that when I started learning how to use Sigil, which is why I'm surprised to find an epub, an opf and a jpg image in the folder I made for conversions. I don't think that was supposed to happen, especially since I chose to save only epub to disk.

To me it seems rather organized because the Calibre Library is one folder.
But it isn't - it's one folder for the library, yes, but then you go several levels deep, with genre folders, author subfolders, subfolders for each and every book... I know that complaining won't do much god because the program can't be changed. I remember reading somewhere in one of the threads that too many subfolders can impair Windows performance so that's also an issue. The developer was surprised by that.

I don't know if I will eventually start using it for management as well. I'm taking it slow this time; first time I imported my entire library and ended up with a mess on my hands. I like the idea of adding summaries. I don't bother with pairing since 90% of fics I save are gen. I don't care for word count, don't separate fics by challenges, the fic can be rated low but I still keep it because it's guilty pleasure/for sentimental reasons, and I simply don't download fics that hit my squicks so warnings aren't important to me when it comes to re-reading. I do pay attention to them when I'm deciding whether or not to check a fic. So fandom, author, fic is pretty much the only thing I need. Read/unread I could apply to new fics, since the ones already saved have all been read many times before. In the end, I guess it's a question of stepping over that mental hurdle of dealing with folders to dealing with metadata. My level of discomfort tells me I'm not ready yet.

[amalthia]

I discovered that when I started learning how to use Sigil, which is why I'm surprised to find an epub, an opf and a jpg image in the folder I made for conversions. I don't think that was supposed to happen, especially since I chose to save only epub to disk.

Yeah that's odd...not seeing your work flow it's hard to say what could have gotten mixed up. I don't use Sigil so I can't say how it works in terms of working files. But generally speaking you should only have the epub where you save your story to. Did you do any tests with saving the converted story somewhere else in a new clean empty folder to see what would happen?

I don't know if I will eventually start using it for management as well. I'm taking it slow this time; first time I imported my entire library and ended up with a mess on my hands.

Same here! :)

I remember reading somewhere in one of the threads that too many subfolders can impair Windows performance so that's also an issue. The developer was surprised by that.

I thought you had to go down about twenty levels before anything would happen. I do know that some programs can't handle more than 40 characters in a file name. But I've never run into any troubles in the three years I've been using calibre. And since Calibre is the only program accessing any of those folders it's all good. :)

Read/unread I could apply to new fics, since the ones already saved have all been read many times before. In the end, I guess it's a question of stepping over that mental hurdle of dealing with folders to dealing with metadata. My level of discomfort tells me I'm not ready yet.

I think for myself I just love that the program does have custom columns. It took me a while to get used to the program too. Hopefully, they'll either come out with a good alternative or the folder structure will stop bugging you.