cURL cannot follow redirects when open_basedir or safe_mode is enabled

Luckily we live in a time where the PHP safe_mode is deprecated.

However some legacy webspaces still have this feature, and also open_basedir is often active.

When using cURL this may be some kind of a bummer, because it prevents you from following redirects. This may be due to the fact, that cURL as a native extension would then be able to follow symlinks in the filesystem and access files which it should not be allowed to do.

The problem

You encounter the following error:

curl_setopt_array() [function.curl-setopt-array]: CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set

The solution

You will find many great approaches to circumenventing this feature when it comes to HTTP connections.
This is done by parsing the Location header directly from the returned data and issuing a new request.

On php.net you will find many solutions. While some are broken, many also work. However, I could not find a solution that worked for my problem:

I wanted to follow a redirect on a site that needed cookies and needed a correct switch of the request-method from POST to GET.

Typically the circumventions copy the cURL-handle which makes it loose the cookies. Also they do no reset the request type, as normal browsers do it.

The code

This code worked for me. Hopefully it works for you.

Note that this code is an improvement to the code from zsalab orgininally posted on php.net

function curl_exec_follow(/*resource*/ $ch, /*int*/ &$maxredirect = null, $postfields = null) {
$mr = $maxredirect === null ? 5 : intval($maxredirect);
if (ini_get(‘open_basedir’) == ” && (ini_get(‘safe_mode’) == ‘Off’ || ini_get(‘safe_mode’) == ”)) {
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, $mr > 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, $mr);
} else {
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
if ($mr > 0) {
$newurl = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);

curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FORBID_REUSE, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

do {
curl_setopt($ch, CURLOPT_URL, $newurl);
$header = curl_exec($ch);
if (curl_errno($ch)) {
$code = 0;
} else {
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($code == 301 || $code == 302) {
preg_match(‘/Location:(.*?)\n/’, $header, $matches);
$newurl = trim(array_pop($matches));
curl_setopt($ch, CURLOPT_POSTFIELDS, null); //also switch modes after Redirect
curl_setopt($ch, CURLOPT_HTTPGET, true);
} else {
$code = 0;
}
}

} while ($code && –$mr);
if (!$mr) {
if ($maxredirect === null) {
trigger_error(‘Too many redirects. When following redirects, libcurl hit the maximum amount.’, E_USER_WARNING);
} else {
$maxredirect = 0;
}
return false;
}
curl_setopt($ch, CURLOPT_URL, $newurl);
}
}

return curl_exec($ch);
}

Note

Your first option should however be to fix the PHP settings, as safe-mode is more of a safety issue then a help. Also open_basedir is not the best option either.

The code I posted here slows down the request by 50%. So use it only if absolutely needed.

You can implemented as standard though, as it has a fallback to use the native FOLLOW_LOCATION feature if possible. (TXH to zsalab)

Whats going on at Yahoo Sponsored Search? Yahoo Ads showing on Google?

Recently I made a new campaign using the german Yahoo! Sponsored Search.

What I did expect is the Ad to be delivered on de.search.yahoo.com as well as on bing.com, cause this is what the Search Alliance is all about.

As well all love it on Yahoo, the Ads cannot be blocked from showing on weird third-party sites, who Yahoo! thinks are interesting for my campaign. Google allows the restriction to only Google Search.

Have a look at the screenshot and see what I mean. Do you think the campaign I’m running on Yahoo! is interesting for customers on erkaltung.com (German for common cold) AND consultdomain.de? And why on earth is my site showing on buzzdock.com although the campaign is targeted for Germany? Probably, we will never know …

 

Yahoo! Sponsored Search Metrics

Yahoo! Sponsored Search Metrics

However, what confused me the most is that Google.de is showing as a URL where the Ad got delivered. I do know about the Yahoo-Google Advertising Agreement, but I thought this only means that Ads from Google can get delievered on Yahoo! not the other way round?

Does this mean that I can place the over-long ads from Yahoo! on Google Search?

Post a comment under this post if you got any idea why this is happening.

- Arne

Switched from BlackBerry to Android – Sorry to all you BlackBerry guys …

Hey Folks,

recently I changed my mobile from ye olde BlackBerry 9780 to the brand new Samsung Galaxy Nexus.

First of all, the mobile is totally awesome and so far I only had positve experiences. No crashes, perfect app-integration and the OS runs very smoothly.

Sadly this means for you, who waited for the Wunderlist for BlackBerry App, that I am not doing anymore development for BlackBerry.

Hope the guys at Wunderkinder are making good progress on their native app and make it available to everyone out there soon.

 

- Arne

Progress on Wunderlist for Blackberry …

Hey Guys,

just wanted to keep you up to date, that the Wunderlist project for Blackberry has not been droppped.

Currently I am facing major issues in deploying the application the Blackberry. It seems to me like the CodeSigning-Servers from RIM are still not really working as they should.

Only 1 out of 10 signing attempts works in my virtual machine and I currently resigned to develop further using my Virtual Machine.

Only 1 out of 10 attempts can complete the request to the signing servers

Only 1 out of 10 attempts can complete the request to the signing servers

In November I will be getting my new Windows 7 Machine and will restart working on this project. Till then it has to be freezed, cause at the moment it is the pure HORROR to develop for Blackberry.

Why do I have to sign an application that I want to run on my own phone, and why is it only possible to work under Windows. RIM I am telling you, if you make it so hard for developers you will definitely loose you place in the market.

Hopefully development will go easier on my Windows Machine in November. I will keep you guys up to date.

 

Best Regards,

Arne

Using Wunderlist with RIM Blackberry

Maybe some of you are using Wunderlist on OS X or on any other supported platform.

So am I, and I can just say that I love it since it is free, and is very good for implementing GTD.

My last GTD tool, iGTD has really gotten a bit old, since its Sync options are totally weird. Syncing with my iCal and then syncing to my Blackberry Bold 9780 never really worked. Sometimes I got the appointments and tasks twice, sometimes they were in the wrong calender and so on …

Wundelist does a great job, but it is still lacking of Blackberry support. Since I love Blackberry and certainly don’t wanna move to an iPhone or even pay for it, I decided to develop my own App that runs on Blackberry and can sync my Wunderlist data.

 

Until now, only the roadmap has been set, but it is fairly straight-forward. If you wanna download the plugin already … well you are quite to early. But check back later, or catch my RSS and you will be informed when the plugin is ready. You can also post a comment to this post, and I will eMail you when it is ready.

For all you interested guys, here is how the plugin will work:

  • Syncing the wunderlist.db via Dropbox
  • Reading the wunderlist.db via native HTML5 database support
  • Blackberry integration via PhoneGap

There are quite some steps to go, but the proof of concept has already been done. In the first version that I will be releasing next week, the plugin can only read data from the wunderlist.db. In later versions you will be able to make new notes to your Inbox.

Implementing the whole category, tag aso. stuff will never be implemented by me, cause I think this functionality does the trick when you are on your mobile. The organization of the tasks can still be done @ home.

Best,

Arne

Ported the old entries from MODx Revolution

Hey guys,

just wanted to inform you that all the old entries from MODx Revolution are now ported to WordPress.

I did not retain the links, nor did I redirect them them via 301 to the new locations … “How do you like that Google!”

I really love it if you know how to do it better, but you are just to lazy. Since I do not run a SEO blog, I think I can handle the trust Google looses in my site by just killing the links ;)

Best,

Arne

Configure SMTP-Auth on exim4

Ever tried to get STMP-Auth running on your own exim4 instance?

Well it is really not much of a problem if you are running Debian and have access to the infamous internet.

This post is basically a copy-cat of the great post from debian-administration.org on HowTo Setup Basic SMTP AUTH in Exim4

The post was a great help, but afterwards my server was still not accepting my SMTP request to send an email. Some people in the comments complained that it was still not working for them, but since the post is inactive for more than a year I decided to post it on my blog.

So lets start …

We assume you have exim4 running, all mails get delivered to the correspondig home-dirs and you can access your server via STMP (PORT 25) without SSL or TLS to send an email to a non-relayed host. (Means to a local mail recipient).

I will copy now the steps from debian-administration.org in case the posts is going offline …

We need to generate a self-signed SSL-certificate by calling

 

/usr/share/doc/exim4-base/examples/exim-gencert

Be sure to add the certificate to your keychain once you connect later on.

Then go to

/etc/exim4/conf.d/auth/30_exim4-config_examples

and uncomment this whole bunch

# plain_server:
# driver = plaintext
# public_name = PLAIN
# server_condition = “${if crypteq{$auth3}{${extract{1}{:}{${lookup{$auth2}lsearch{CONFDIR/passwd}{$value}{*:*}}}}}{1}{0}}”
# server_set_id = $auth2
# server_prompts = :
# .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
# server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
# .endif

and this whole bunch

# login_server:
# driver = plaintext
# public_name = LOGIN
# server_prompts = “Username:: : Password::”
# server_condition = “${if crypteq{$auth2}{${extract{1}{:}{${lookup{$auth1}lsearch{CONFDIR/passwd}{$value}{*:*}}}}}{1}{0}}”
# server_set_id = $auth1
# .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
# server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
# .endif

These steps will enable you to login via auth plain and auth login. Depending on your eMail program you may need the one or the other. It is save to enable both, your program will choose the correct automatically.

Then the tutorials says to add the line

MAIN_TLS_ENABLE = true

to the file

/etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs

This configuration is correct but it makes debugging HARD, since now your server does not respond with 250-AUTH plain when you do an EHLO localhost via telnet on your server. You first have to do a STARTTLS or use openssl in the first place ;) .

Before we de create a new user via

/usr/share/doc/exim4-base/examples/exim-adduser

and the restart via

update-exim4.conf
/etc/init.d/exim4 restart

Now we connect through openssl by calling this command

openssl s_client -host my.server.name -port 25 -starttls smtp

 

and everything should be working fine.

If you receive the error like 435 Unable to authenticate at present” then maybe exim4 cannot read your passwd file under /etc/exim4/passwd for debugging try to set it to 777, but if it works set it to the correct value, according to the group exim4 is in.

 

A good german post on testing SMTP-Auth with telnet is on computer-tipps.info : Testing SMTP with Telnet

 

Back on WordPress Again!

Hey Folks,

after a long experience on my once favored CMS MODx I returned to WordPress.

You are curios why? Well I really loved MODx because of its flexibility and its enormous potential to quickly integrate PHP Code as a very modular plugin.

I still got some Evolution Codebases running for some of my clients, but I really never installed Revolution for any of my clients.

Maybe you have read one of my older posts, where I talked about porting my Blog to Revolution. It sure was no easy task, and the performance at the end was frustrating.

Installing plugins was now made through a totally buggy package transport system etc.

 

Then today I ported all my websites from a Shared Hosting to a vServer and also “tried” to update MODx Revolution to its newest version 2.1.3. Well the task did cost me more than an hour, some serious headaches and quite some coffees.

MODx did not finish the update correctly, by NOT altering some tables. The result was a broken installation, which could not be reverted.

After replaying my backup I tried the whole routine again, cause I read in … that it was maybe some issue with the MySQL user privileges.

Well this post did not change a thing. Same error in the log files:

 

Unknown column ‘modUser.hash_class’ in ‘field list’

Alright, now another post led me to the issue, that it could be something with wrong CHMOD prefs.

I set all the directories and files to 777 and reinstalled again. Et voilá, …. still nothing, but the error changed, hooray.

PHP Fatal error: Call to undefined method modX::getUserDocGroups()

 

Well what is this?! A fatal error in a fresh install? Luckily MODx own bugtracker had this one filed under : Bug #5451

The best part is “The solution” :

- uninstall Wayfinder (2.1.3 at the time of revo 2.0.8)
- redownload it as “new package” as package manager does not(!) recognize the newer package (says “no updates for wayfinder”)
- install Wayfinder 2.3.1

Wow super, I got a broken install here and I should change something in the OLD UNBROKEN VERSION! Seriously guys?

Not that it was impossible to revert the whole install again and then change the Wayfinder setting, but after nearly 2 hours I just had enough. I haven’t yet installed the system and the errors were not stopping. My crystal ball tells me that there are more problems coming up …

So long MODx, we had a fair time together.

 

For all you guys that still want to continue I made some “shownotes” to dig further.