WordPress is a great tool and you can hack all sorts of functionality into it, but have you ever thought about using it as a user authentication database for content on your server that is outside the realm of WordPress? Maybe a wiki or media server application that you only want your registered WordPress users to access.
There are some really awesome authentication tools built right into WordPress that you can use verify a username and password within your WordPress install. You can even look at that user’s specific capabilities to determine if they get access or not based on their role or capabilities.
In the following example, I use PHP’s ability to present the user with a basic HTTP authentication dialog box, and then it’s authenticated against the WordPress database.
Only thing I haven’t done is set a cookie that keeps them logged in across browser sessions.
This works great if you’ve got an application that has a rewrite to a single index.php file to serve everything, or else put it into a header file that gets served on every page (above any HTML output since it sends our HTTP headers).
And remember: this security is only as good as WordPress security–which is to say “not very secure” but it sure beats an internal non-password protected server that anyone could access simply by plugging into your physical network and browsing around.
cURL is a fantastic way to scrape data from websites. It’s pretty ubiquitous on LAMP servers nowadays, so you probably don’t even have to do anything to enable it and start using it. You can essentially get data that’s behind a login form by spoofing a browser logging into the site.
I’m not going to do a dissertation on the nuances of using cURL. Instead, I’d like to discuss how to process data after you’ve gotten the HTML string from a page using the curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); setting. The general process using cURL to get the data from a password protected site goes something like this:
Make sure your “cookie jar” is set up and working so that your session can be saved while you’re scraping other pages after login.
Get the HTML for the login page.
Get the form post location.
Get any special tokens or hidden variables.
Post to the login script with your authorized username and password.
Scrape any pages you need to get and process them.
Getting auth tokens and stuff
Sometimes login pages make it difficult to figure out what to post because of “authentication tokens” or similar variables that change every time a page is hit and then have to be posted back along with the username and password. This is most common among .NET applications since it’s built right in to the .NET form creation classes. The process of finding these variables is made a lot easier if you use phpQuery. It’s a project that attempts to replicate what jQuery does but with PHP. You pass phpQuery a string with all the HTML content in it and then you can perform selectors and traverse the DOM just like with jQuery. What you do with the HTML after that is up to you.
In the above example, I already had the HTML string of the login page and simply used phpQuery to get all the inputs on the page with a very jQuery-like selector syntax and then looped through the results to get all my input fields for the form so I could use them in my next call to post the data back to the login script.
Getting poorly formatted data and stuff
I consider “well formatted” data to be in formats like JSON, XML, CSV, etc. that are specifically meant for data transfer. But what heppens when you need to scrape data from HTML tables or DIVs?
phpQuery to the rescue! You can do the same thing as above but parse the newly acquired HTML string of the data you’re looking for. Here’s an example of looking at a two column table of data to get all sorts of neat stuff and format it into an array. Then I check for an element outside of the table loop and set another variable.
Using the data and stuff
After you’re done with that, you have all this useful data in an array and you can basically do whatever you want to it. Since this example I created originated from code used in a real life client project (with variable names changed to protect the innocent), I went on to take that array and save it out to a JSON file that I could easily read in and do what I wanted to with the data.
You may need to go through some tests to get the right kind of cleaning or data sanitization for your values, but phpQuery makes it really easy to scrape this kind of data if you’re used to jQuery’s selectors and traversing. Instead of scraping the site each time to get the data, I like saving out the html string to a file and then playing with selectors and traversing to get the data I want form a local file so I don’t put a lot of strain on the server.
And don’t forget stealth…
If you’re not sure how friendly the server (or serveradmin) is to you doing this, make sure you set the cURL user agent for each request. I like pretending I’m GoogleBot, but any common user agent string would suffice so that the server doesn’t explicitly know you’re PHP trying to log in.
By default, cURL spits out a user agent like this (changes based on operating system and version of cURL):
If a serveradmin sees that in their logs, they might freak out… so I like these user agent strings for sneaky data pulls:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
Audience participation
How do you use cURL and phpQuery? Any useful scripts or resources to point people to other than their respective documentation sites?
Apple has allowed iOS apps to register their own URL schemes on your devices for a while now, but I’ve never used this functionality in-depth ’till just recently.
Having your app register a URL scheme on a device means that you can open an installed application on a user’s iPhone or iPad in ONE CLICK from their email (or the web, or another application). There are all sorts of apps already using this functionality.
Use Case
We just used this for a project where the client would send confirmation emails to the user after signup. The email would contain an link to a URL. The URL opens in mobile Safari, then launches the app and seamlessly logs the user into the app.
That’s one use case; there’s probably others (let us know yours). Another one could be a URL embeded in text—say someone is reading on their iPad and the content could prompt them to hop over to the app.
Try It Out
If a user has your app installed on their device, all you have to do is send them to my-great-app://whatever/ and Safari will resign to the background and your app will open. Why don’t you try it? I know you’ve got the Facebook app installed on your iPhone. Just clear your address bar and type (or click the link): fb://profile
Go ahead, I’ll wait.
It opened the Facebook app and took you to your profile page (or gave you a big fat cryptic error message if you don’t have the Facebook app installed). Facebook allows a LOT of options that you can pass to the application to take you into various parts of the app.
Process
So enough about the Facebook app. What does a developer do if they want to pass a bunch of data to the app—without actually defining what needs to be passed using a bunch of if/else or switch statements?
It’s fairly trivial to add url scheme to your application’s Info.plist:
Register the url scheme in your Info.plist. Lets say: my-great-app://
In the application delegate, implement this method and return YES:
When the user clicks a link like that, or you auto-redirect them from your site to that URL, you can parse the URL into an NSDictionary with this code. Here’s the method and an example implementation along with some pretty badass comments so you know what the heck is going on:
Let me know in the comments if you find this useful at all or if you make changes to it.
If you’ve ever taken a gander at a string sanitization class or library, you’ve probably noticed the amount of code necessary to keep the script kiddies at bay. We’re talking about slow string manipulations, i.e. string replacements and regular expressions.
Most MVC Frameworks today come with some form of sanitization or input filtering class built in. The problem with many of these libraries is that they fail to clean some of the more creative attack vectors. To combat this, some people use drop in libraries like OWASP AntiSamy or HTML Purifier to ensure their data is getting scrubbed clean. HTMLPurifier, for instance, uses a Smoke Test via the ha.ckers.org XSS attack list (the de-facto standard for finding attack vectors to test, might I add) to ensure they’re cleaning anything and everything. Continue reading →
As part of our ongoing programming R&D at Skookum Digital Works, we’ve recently been experimenting with WebGL. If you’re new to the technology and 3d in general, it can be quite a daunting task to get up to speed. Here’s a quick WebGL primer (along with Three.js) to get you started.
WebGl is not your Grandma’s Javascript.
WebGL is awesome. It is a very low-level api that extends JavaScript to allow the generation of 3D graphics via the canvas element. The cool thing is that it actually uses the power of your gpu to render the graphics, allowing video-game quality results without any plug-ins. Its important to know that this is bleeding edge tech, the 1.0 specification was just released in March which leads us to…
Browser Support
As you can see by the above chart, all of the “cool” browsers are WebGL ready. (Safari does require support enabled via the command line). Then of course we have IE. Sadly the folks at Redmond decided not to include the tech in their latest browser. Hopefully things will change with IE 10.
Not for the Faint of Heart
A drawback (of sorts) is that WebGL is basically a wrapper for openGL. The good part about this is that the possibilities are endless. The bad part is that if you are not familiar with 3d/game development then the learning curve is quite high. Here is a brief run-down on the process for creating a cube in straight WebGL:
Get canvas element with webgl context
Set viewport
Configure depth buffer
Create perspective matrix
Create modelview matrix
Create and compile shader(s) and shader program(s)
Create vertex array
Create and bind vertex buffer…
View the source on the following demo to see what I mean.
All that for a white box? Yep. Believe it or not that “box” is actually an unlit cube in 3d space, and as you can see it is no trivial task setting things up to render it.
Introducing Three.js
Theres got to be an easier way! Well there is, the answer is Three.js. This WebGL 3D engine from mrdoob is EPICALLY AWESOME! In a nutshell it allows you to easily render 3d scenes with a minimal amount of code, allowing you to focus on the fun stuff. Look at it like jQuery for WebGL.
Check out how much shorter our code is recreating that first demo with three.js
The other cool thing about Three.js is you can pick from a 2d canvas or svg renderer as well. The performance will be different and not all cool 3d api options will be available, but its good knowing that you have the option if you want to create something basic with maximum browser visibility.
The Fun Stuff
Ok enough with the boring boxes. Here is a demo where we dynamically add spheres to the scene, and then rotate that scene based on the coordinates of our mouse.
As you can see the code is not much longer, and plays nice with others libs like jQuery and sugar. Remember to animate using the requestAnimationFrame instead of setInterval for maximum efficiency.
The Grand Finale
Now if you are still craving more (I know you are). Just take a look at this last demo. It is a modified version of the great app from airtight.cc called the Rutt-Etra-Izer. I have tweaked it to allow the selection of a Skookum’s twitter account. Once the account is selected it will pull in that person’s main profile picture and perform the effect. The z-depth of the lines is determined by the lightness or darkness of the pixel. And you can see the rendering speed of Three.js/WebGL by adjusting the parameters in real time at the left. Have fun!
