Quick emerge --sync for Gentoo
Switch to a different DB back-end:
# vi /etc/portage/modules
portdbapi.metadbmodule = portage_db_anydbm.database
portdbapi.auxdbmodule = portage_db_anydbm.database
eclass_cache.dbmodule = portage_db_anydbm.database
Then do 'emerge –metadata'. The first time, it will take a long time to build the anydbm databases for every single ebuild in Portage. After that, syncs will be super fast and take only seconds instead of minutes.
Also, searches will be MUCH faster, too.
If you want to stop using anydbm in the future, just delete these three lines or the entire /etc/portage/modules file, and then do 'emerge --metadata'.
6 Replies
You might avoid that long wait by doing 'rm -rf /var/cache/edb/dep/usr/portage' before doing the emerge --metadata.
Or you can cheat and tar up /var/cache/edb/dep/usr/portage/*.anydbm from any other Gentoo/x86 machine with latest anydbm db files into a tar file and scp that to your Linode, and unpack. Then do 'emerge --sync' to update the rsync'd over files.
Then you're set, either way.
Seriously amazing to see how fast the end of emerge --sync runs after that first time. Finishes the db update in like 40 seconds instead of 5-15 minutes.
There are a few drawbacks to that script though. But at least you won't have KDE or Gnome getting in your way (assuming you haven't installed either).
Most likely I misunderstood the purpose of this change. I've just tried it on my home machine - the
emerge --metadata
took about 1h30. The
emerge --sync
was very slow doing 'Updating Portage cache'. I interrupted it on 27% after 16mins . I switched back to the original setup and the sync took under 2 minutes.
I'd assumed changing the backend would speed up the Update Portage cache. What am I missing here ?
Thanks,
Graham.
I've been trying to figure this one out for a few weeks now. (Haven't ignored it)
I can't seem to reliably reproduce it yet, but I do see what you mean. Hmm. Still thinking and debugging.
The error message is:
emerge --metadata
====================================
Error: Failed to import module 'portage_db_anydbm.database'
File "/usr/lib/portage/pym/portage.py", line 1439, in load_best_module
mod = load_mod(best_mod)
File "/usr/lib/portage/pym/portage.py", line 145, in load_mod
mod = __import__(modname)
No module named portage_db_anydbm
====================================
a little info:
emerge --version
Portage 2.1-r2 (default-linux/amd64/2006.0, gcc-3.4.6, glibc-2.3.6-r4, 2.6.17-gentoo-r4 x86_64)
What could be wrong here?
To make a long story short… you wouldn't need this stuff now, because Portage 2.1 is now stable. 2.1 dramatically speeds up that stuff without needing any special hacking.
2.1 also changed internal APIs, so old Portage 2.0 backend db modules no longer work 'as is'. This is probably why you are seeing these errors.
My strong recommendation? Do not implement the stuff I originally suggested in the first post in this topic. No longer needed.
emerge --metadata (which is a part of emerge --sync and done automatically when you sync) just zooms by on my Linode, every time, with Portage 2.1 in place!
It looks to be dramatically more effective in not touching the database for any directories that has not updated since the last sync/metadata update.
If you have /etc/make.profile pointing to any 2006.0 profile and if you have done 'emerge --sync' within the last two months, then you should now have Portage 2.1 installed.
To check:
# equery list -e portage
[ Searching for package 'portage' in all categories among: ]
* installed packages
[I--] [ ] sys-apps/portage-2.1-r1 (0)