For the two other people in the world who might run into this problem:
I’m trying to install NextCloud on a fresh Ubuntu 18.04 system, and I want to use the user_saml app and Shibboleth for user authentication. This means I need to use php7.1, first off, because 7.2 doesn’t have mcrypt support and user_saml requires it. I’d also like not to have to build everything from source. The problem I’ve run into is that php7.1-curl depends on libcurl4 and libapache2-mod-shib2 depends (through libxmltooling) on libcurl3, and you can only have one of those installed at a time.
I have found a workaround, and I’m documenting it here mainly so I don’t forget how it works, not because I expect anyone else will find it useful or interesting.
First, install libcurl3, with “apt-get install libcurl3”.
Once that’s installed, go to /usr/lib/x86_64-linux-gnu/ and make a copy of libcurl.so.4.5.0, using “libcurl3” as the name instead of “libcurl”. You should end up with something like this:
-rw-r--r-- 1 root root 518600 Mar 5 04:41 libcurl3.so.4.5.0 lrwxrwxrwx 1 root root 19 May 8 10:47 libcurl-gnutls.so.3 -> libcurl-gnutls.so.4 lrwxrwxrwx 1 root root 23 Jun 1 18:04 libcurl-gnutls.so.4 -> libcurl-gnutls.so.4.5.0 -rw-r--r-- 1 root root 510408 May 8 10:47 libcurl-gnutls.so.4.5.0 lrwxrwxrwx 1 root root 16 Jun 1 18:04 libcurl.so.4 -> libcurl.so.4.5.0 -rw-r--r-- 1 root root 518600 May 8 10:47 libcurl.so.4.5.0Install libcurl4, which should remove libcurl3, but leave the version 3 file you just copied there: “apt-get install libcurl4”
Double check that the libcurl* files in /usr/lib/x86_64-linux-gnu are still pointing to the right places — none of the symlinks should be pointing to the libcurl3 file!
Next, download the package for libxmltooling7, which is the one which depends on libcurl. We’re going to repackage it without that dependency, so apt won’t complain later: “apt-get download libxmltooling7”
Unpack the .deb: “ar x libxmltooling7_1.6.4-1ubuntu2_amd64.deb”
Unpack the control tarball: “tar xf control.tar.xz”
Edit the file “control” and change “libcurl3 (>= 7.16.2)” to “libcurl4”. Remove the version requirement. You could probably also just remove the libcurl3 bit entirely, but might as well let any libcurl4 satisfy the dependency.
Repackage the control tarball: “tar -cJvf control.tar.xz control md5sums shlibs triggers”
Repackage it all into a new .deb: “ar rcs libxmltooling-local.deb debian-binary control.tar.xz data.tar.xz”
And install the fixed libxmltooling: “dpkg -i libxmltooling-local.deb”
Now you should be able to install shibboleth, but first let’s have systemd load the libcurl3 library when it tries to launch shibd during installation. (This is the bit which took me the longest to figure out, because while I knew LD_PRELOAD ought to work, I didn’t realize systemd was sanitizing the environment and preventing it.) Run this command: “systemctl edit shibd”
If that doesn’t work because there isn’t a shibd service yet, just make a new file named “/etc/systemd/system/shibd.service.d/override.conf” and edit that. Into that file, paste this:
[Service] Environment="LD_PRELOAD=libcurl3.so.4.5.0"Save the file, and now try installing shibboleth: “apt-get install libapache2-mod-shib2”
With any luck, you’ve now got a system with libcurl4 and a working shibd linked against libcurl3.
Now you can (re)load all the php modules, and they shouldn’t complain about dependencies or try to remove any of the shibboleth packages.
The dependency of libxmltooling7 on libcurl3 will actually bite anyone who wants to run Shibboleth on Ubuntu 18.04 alongside any of the hundreds of packages that depend on libcurl4. I’ve submitted a bug report to the maintainers of the Ubuntu xmltooling package: https://bugs.launchpad.net/ubuntu/+source/xmltooling/+bug/1776489
Awesome; I’d much rather have that conflict resolved officially. Especially because it looks like some background process keeps re-linking libcurl.so.4 to the libcurl3 library, breaking everything but shibd.
It looks like maybe if I put the libcurl3 library in its own directory and add that directory to /etc/ld.so.conf(.d/shibd.conf), I can add the full path to the LD_PRELOAD declaration and have shibd use it there, and not have ldconfig rewriting the symlink for the system libcurl.
Update: Putting the libcurl.so.4.5.0 from libcurl3 into /opt/libcurl3 works as long as you name the ld.so.conf.d file such that it alphabetizes after x86_64-linux-gnu.conf
Here’s my working config:
If you just name that file “libcurl3.conf”, it gets loaded before x86_64-linux-gnu.conf, and now everything linked against libcurl will try to use the libcurl3 version. So make it load last and then specify the full path to it in the systemd override file.
Hi Josh,
Thanks a lot, you saved my day :D.
Just some small changes to script:
Repackage the control tarball: “tar –xz -cvf control.tar.xz control debian-binary md5sums shlibs triggers”
Should be:
Repackage the control tarball: “tar -cvJf control.tar.xz control md5sums shlibs triggers”
And Some links to the bugs, for everybody interested:
https://bugs.launchpad.net/ubuntu/+source/xmltooling/+bug/1776489
https://issues.shibboleth.net/jira/browse/CPPXT-110
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828607#49
https://bugs.launchpad.net/ubuntu/+source/curl/+bug/1754294
Good catch; thanks!
Seems like there’s now 3 people in the world who’ve run into this issue.
I’m from ULYSSIS, a student organisation at the KU Leuven in Belgium. We use shibboleth-sp to integrate with our university’s IdP and offer all kinds of services to students, including official voting and election tools. We also ran into this problem with shibboleth while upgrading from 16.04 to 18.04. On top of that, shibd now also runs as _shibd instead of root, which gave us problems with the CredentialResolver key, since it was only readable by root.
We’ve plain dropped the libcurl dependency, since we couldn’t install libcurl4 since this is an upgrade instead of a new installation.
We’ve also started a discussion about the bug https://bugs.launchpad.net/ubuntu/+source/xmltooling/+bug/1776489 behind this problem on a few IRC channels, and hopefully this can be solved with an SRU.
Thank you for sharing this information, it really helped us find out what was even wrong.
What a pain!? Thanks for the info on getting this working.
Not sure how they’ve ignored it for so long. In addition to the bug Bert mentioned above, there was this bug reported last March: https://bugs.launchpad.net/ubuntu/+source/curl/+bug/1754294
That bug impacts 151 people, so it’s definitely more than 2 (or 3, or 4 …). I’m wondering if Universities can chase this up via Internet2 to put some additional pressure on devs to get this fixed.
A couple of notes on the instructions:
1.) You may want to add an additional step of setting libxmltooling7 to not update. I ran an apt-get dist-upgrade right after these steps, and apt kindly removed libcurl4 again. You can get it to stop that by running “apt-mark hold libxmltooling7”
2.) When you force apt to install libcurl3, it will also remove some other packages/meta-packages. Not many, but a few. After reinstalling libcurl4, you may want to add a step to run “apt-get install ubuntu-server” to reinstall all of the missing packages. Then have people double check the symlinks. I think my symlinks broke again after it tried to reinstall curl … but IDK … I wasn’t paying that close of attention.
3.) Super minor here, but adding the command to fix the symlink, “ln -fs libcurl.so.4.5.0 libcurl.so.4” would be really cool as well.
I don’t want my notes to sound petty … these instructions were awesome, and got me 95% of the way there! Good work!
I think the simplest solution is to use the apt repo provided by switch
https://www.switch.ch/aai/guides/sp/installation/?os=ubuntu
I lost count, am I #5 on the impact list?
Thank you for this – saved me hours (days) of aggravation!
Thank you so much for making this excellent guide. I now have my predecessor’s custom Shibboleth up and running again after a server migration. Everything seems to be working.
I just wanted to say that I’m one of the two people in the world that needed this. This was to get the shibboleth sp working with the Redcap survey system. Thank!
https://www.project-redcap.org/
Just wanted to add to the chorus of thank-yous! I think I must be #5 to find this wildly helpful! Like others I made a few modifications; in my case, it was because I was hoping to avoid downtime on a server which relied on libcurl3/libxmltooling7 for its shib install but that needed libcurl4 to upgrade its ceph version.
Here’s what I did: first, I made the copy of libcurl.so.4.5.0, then I rebuilt the libxmltooling7 .deb without any curl dependency (so it wouldn’t uninstall my shib utils, rendering the site unusable), then installed libcurl4. After that I rebuilt libxmltooling7 with the libcurl4 dependency (and marked libxmltooling7 on hold, thanks to Daniel’s tip). After that, I was able to upgrade the ceph install.
So overall, I managed to do all this on a mission critical server with zero downtime. It was pretty fancy, and I’ll be forever grateful for your steps to help me get there!