From Media Mover project page

Media Mover is a set of modules which allows website administrators to easily create complex file conversion processes.

Media Mover is essentially a system for Drupal that is targeted at file harvesting/conversion/storage or all three.

Just like the harvesting the crops, harvesting, in this context, means collecting files from certain location - be it FTP, email or a filesystem location - and processing them as per the set configurations to achieve a certain result.

For example, Media Mover can scan an FTP location for files, bring over the files and attach them to existing nodes or create new nodes based on the configurations.

At core of the module is media_mover_api module that acts as the handler for the above stated process. It defines four types of behaviors for how modules interact with a file: harvest, process, storage, and complete. A configuration can be made up of multiple modules which define different kinds of harvest, process, storage, and complete behaviors. For example, you can have two different processes that creates high and low bandwidth versions of a video file.

Media Mover allows administrators to setup multiple configurations that are responsible for different kinds of media handling. You can set up as many different kinds of configurations as the each configuration is distinct.

Media Mover API also allows creation of modules that can use this API to create your custom harvest/process/store functionality.

Media Mover supports follow operations in the core module:

• Harvest: get attachments from an email account
• Harvest: get attached files from nodes of X type
• Harvest: get previously harvested files
• Harvest: get a file path from a CCK field
• Harvest: get files from a FTP server
• Process: create a thumbnail of a video file with ffmpeg
• Process: transcode video files into web friendly (and other) formats
• Process: strip EXIF data from images (MM EXIF)
• Process: add meta data to a FLV file (MM FLVTool2)
• Storage: move a file to Amazon’s S3 service (this also operates as a completion function)
• Storage: attach file back to node
• Storage: save file and related data as a node
• Storage: add a file path to a CCK field
• Storage: save file as an Asset

Media Mover is currently available only for Drupal 5.x. According to its project page, Drupal 6.x support is in development and soon a development version should be expected.

In the mean time, take a look at its Media Mover API docs(if you are a developer). I have not covered its usage because its UI is straight forward and should not be difficult for those who have basic experience with configuring Drupal.

Since this is the first post in DrupalMOTW series, I could not think of a better module than Drush. Drush has been making my life easier for past 6 months and I must confess that I have fell in love with this module and I am sure you will also fall in love like me(with Drush)

From Drush project page:

drush is a command line shell and Unix scripting interface for Drupal, a veritable Swiss Army knife designed to make life easier for those of us who spend most of our working hours hacking away at the command prompt.

Let’s take a look at what Drush provides.

Beyond this point it is assumed that Drush is installed. I am not going to cover installation of Drush as it is very straight forward, like any other Drupal module. Download, Extract, Enable and you are ready.

Drush comes bundled with a couple of modules.

• Drush
  This is the core module that provides the base for other Drush modules to build on top of.
• Drush Package Manager
  Drush package manager allows you to install and update modules from the command line. However, this module does not have the capability to download the modules in specific location. You require package handlers for that purpose. Drush bundles two package handlers.
• Drush Package Manager CVS support
• Drush Package Manager wget support
  These two packages provide the actual support for downloading the modules. if you are using wget support then it will download the latest released packages using wget(wget is a command line download manager and comes installed with most Unix/Linux flavours) and using CVS support, that will checkout the modules from their respective repositories. I generally prefer using CVS handle,. because with CVS I do not have to wait for package releases to get important updates.
• Drush Package Manager SVN support
  As the name might suggest, this module is not exactly a package handler like CVS and wget versions. This adds support for synchronizing and committing the newly installed/updated modules into svn repository of the project.
• Drush SQL Commands
  Provides several tools for database query and management.

• Drush toolbox
  Provides several tools that are aimed at site administrators and developers.

• Drush SimpleTest Runner
  It runs unit tests on your drupal install.

Using Drush

While single drush install can be used to administer multiple Drupal sites, I generally prefer using separate drush installs for each drupal project I am working on. After installing drush, I generally create a link to drush command line script to the root of my drupal project directory

# something
$ cd ~/projects/drupalproject.com
$ ln -s sites/all/modules/drush.php ./drush

Now when you run ./drush and you will get:

$ ./drush [command]
Usage: drush.php [options] <command> <command> ...
 
Options: 
  -r <path>, --root=<path>      Drupal root directory to use (default: current directory)            
  -l <uri> , --uri=<uri>        URI of the drupal site to use (only needed in multisite environments)
  -v, --verbose                 Display all available output                                         
  -y, --yes                     Assume 'yes' as answer to all prompts                                
  -s, --simulate                Simulate all relevant actions (don not actually change the system)     
  -c, --config                  Specify a config file to use. See example.drushrc.php                
 
Commands: 
  help                  View help. Run "drush help [command]" to view command-specific help.
  pm install            Install one or more modules                                         
  pm refresh            Refresh update status information                                   
  pm update             Update your modules                                                 
  pm info               Release information for a module                                    
  sql url               Print database connection details.                                  
  sql version server    Print database server version number.                               
  sql version client    Print database client library version number.                       
  sql dump              Exports the Drupal DB as SQL using mysqldump or pg_dump.            
  sql query             Execute a query against the site database.                          
  sql load              Copy source database to target database.                            
  cache clear           Clear all caches                                                    
  cron                  Run cron                                                            
  watchdog show         Show the most recent watchdog log messages                          
  watchdog delete       Delete all messages of a certain type                               
  sync                  Rsync the Drupal tree to/from another server using ssh

As you can see, running drush without arguments will list out possible commands you can run with drush.

Drush package manager
Drush package manager can be used to install and update the drupal modules automatically. As of writing, drush only support installing and updating modules.

# Install a module
$ ./drush pm install project
 
# Install a specific version of module 
$ ./drush pm install  project-1.x-dev
 
#install multiple mdoules at once
$ ./drush  pm install project_1 project_2 project_3

Note that drush package manager will only download or checkout the code from http://drupal.org. It will not enable the module/s. You will have to do them yourself from admin/build/modules

And here is how to update modules to latest version

# Refresh the modules update status
$ ./drush pm refresh
Refreshing update status information ...
Done.
 
# And now update
$ ./drush pm update
Update information last refreshed: Fri, 08/29/2008 - 16:04
 
Update status information on all installed and enabled Drupal modules:
  Name                          Installed version       Recommended version     Status
  Drupal Administration Menu    6.x-1.0                 6.x-1.0                 OK                       
  Drupal                        6.3                     6.4                     SECURITY UPDATE available
  Drush                         6.x-1.0                 6.x-1.0                 OK                       
 
NOTE: An update for the Drupal core is available. 
Drupal itself can't yet be updated by this tool. Please update Drupal manually.
 
drush: No updates available.
 
$

As you can see from the above examples drush has reported that I two modules are installed and and I have SECURITY UPDATE available for Drupal core. If updates are available it will report accordingly and update the stuff.

NOTE: drush will not update your drupal core.

Now lets take a look at what Drush SQL Command module provides us. Code speaks more than words, therefore code below is self explainatory

# get version of SQL server
$ ./drush sql version server
5.0.51a-3ubuntu5.1
 
# get version of SQL client
$ ./drush sql version clinet
5.0.51a
 
# Get the database connection string
$ ./drush sql url
mysqli://username:password:mysql.example.com/database-name
 
# dump the database
$ ./drush sql dump > mydatabase.sql
 
# Run an SQL query
$ ./drush sql query "SELECT nid, title, status, created FROM node WHERE type = 'page'"
nid     title   status  created
1       About   1       1219049818
3       License 1       1219059141
 
# Clear the cache related tables
$ ./drush cache clear
Cache cleared.
 
# Show the watchdog entries (removed some entries due to security/NDA reason)
$ ./drush watchdog show
  Date              	Severity	Type   	Message                               	User     
  08/29/2008 - 13:43	notice  	cron   	Cron run completed (via poormanscron).	Mir Nazim
  08/29/2008 - 13:46	notice  	content	blog_post: updated title.                	Mir Nazim
  08/29/2008 - 14:43	notice  	cron   	Cron run completed (via poormanscron).	Mir Nazim
 
# Delete all watchdog messages
$ ./drush watchdog delete [all|error|notice]  # what ever your message type is

As you you might have understood, Drush is truly an amazing tool and a must have for any serious Drupal developer. You can also extend drush and add your custom commands. It’s as far as implementing a few hooks.

While this post could use another section about extending Drush, I am leaving that out for now because this has already turned into a looooooooong post and I do not want to bore you further.

Based on the similar idea I starting a weekly blog series about the modules available to be used with our beloved Drupal. Now it may not be exactly weekly, but I will surely try to cover a module every week for a near foreseeable future.

Stay tuned

<?php
// File Name: print_file_contents.php
$file = $_GET['filename'];   //get the file name from query string
print file_get_contents($file);   //print the contents of the file
?>

If you have worked with PHP for couple of years you must pick up what the problem is. For those are new with PHP let’s look at some examples.

Assuming this file is served from www.example.com. Now if you are using it to display some html or text files looks like there will not be any problem.

# No problem with following URL.
http://www.example.com/print_file_contents.php?filename=somefile.txt
 
# Again no problem
http://www.example.com/print_file_contents.php?filename=somefile.html

What if some smart ass decides to be a bit adventurous and tries something like this:

http://www.example.com/print_file_contents.php?filename=index.php

This will expose the code in index.php. Now he can see all the code that you vomited out in index.php. You think you are screwed now. No, not yet!. Given that now your index.php is publicly visible which actually means whole of your application is visible to everyone. Even someone with a feeble brain can figure out what files you are including in index.php and paths to them and by simply passing the path of whatever code file he wishes, into a $_GET variable. Even your database credentials are open and accessible. NOW YOU ARE REALLY SCREWED.

Now, you may ask how to prevent it. Consider the same problem, but slightly different code.

<?php
define('FILES_PATH', '/var/www/myapp/files/');   //define a constant with path to files
$file  = $_GET['filename'];   //get the file name from query string
$path_to_file = FILES_PATH . $file   //create a valid path
print file_get_contents($path_to_file);    //print the file contents to browser
?>

You might think the problem is gone and you are secure now. No you are wrong again. The same problem still exists. If someone requests some like:

# Got you again
http://www.example.com/print_file_contents.php?filename=../index.php
 
# Got you again
http://www.example.com/print_file_contents.php?filename=../index.php
 
# Or even worse
http://www.example.com/print_file_contents.php?filename=../../../../../etc/password
 
# please node that original unix password file is 'passwd' and not 'password'.
# For some reasons PHP does not allow mentioning the original path to this file in POST data.
# May be this is for security reasons.

Such an attack is called Directory Traversal Attack. Directory traversal is trickier to prevent than it might seem. A filter out known bad characters protection strategy is likely to fail.

Enough now, let’s see how those attacks can be prevented. Here are the some things you must take care of.

1. Relative paths are dangerous. Do not use them and check for them. Remember relative paths are all paths that contain a .., even if it starts with a ‘/’.
2. Go back to basics. Establish naming conventions and don’t break them and off course validate the names against the proper naming conventions.
3. Check if you are accessing the correct path.
4. Run checks relevant to you application.

Following code example demonstrates above checks in a simple way. You may need more serious checking for your production environment.

<?php
define('FILES_PATH', '/var/www/myapp/files/');   //define a constant with path to files
$file  = $_GET['filename'];   //get the file name from query string
 
// Check #1: Check for relative paths.
if (strstr($filename, '../' )) {
  die ("Na Na Na. Don't Try to be smart");
}
 
// Check #2: Check for naming conventions. e.g we only allow *.txt files.
// I am just checking the extension. You can use regex, if your's is complex naming convention
$ext = substr($filename, -3);
if ($ext != 'txt') {
  die ("Woah! you are smart but I am smarter");
}
 
$path_to_file = FILES_PATH . $file   //create a valid path
 
// Check #3: Check if you are accessing the correct path.
if (FILES_PATH != dirname($path_to_file)) {
  die("Run away! you have no place here");
}
 
// add any additional checks that might be relevant to your application
 
// and then finaly
print file_get_contents($path_to_file);    //print the file contents to browser
?>

I do not claim that the code is 100% fool proof. NO CODE EVER IS. But it is better than have a totally unsecured code.

There is one more thing that can be done in this regard. You can chroot your application, but be careful if the chroot jail is incorrectly created, you are up for ever more surprises(for worse, off course). For example open file descriptors pointing to the files outside the chroot jail.

_IMPORTANT: I hope you understand that above problems can also arise if you where including the files using or instead of output to the browser._

Note for Drupal developers

Since I am a Drupal developer myself, I think I must point out how to be careful with including files when developing for Drupal, Make sure all of your includes are done using Drupal utility functions like:

<?php
drupal_get_path();
drupal_add_js();
drupal_add_css()
?>

This will save you lot of headache later on.

Mother Kashmir, may it be remain prosperous, may it remain happy - Mother Kashmir.

I say What The Fuck.

Suppression and subjugation of Kashmir and Kashmir people is still on. People being killed unprovoked. For past few days, Indian Forces have been killing people by forcing entry in our homes, not even sparing women, infants and children.

And The Blind Indian Media or Blindia has simple shut its eyes on Kashmir.

Three civilians were killed and at least 50 others injured when Police and paramilitary CRPF troopers fired upon the protesters in different parts of the Valley yesterday, witnesses and reports said.”(read the full story on GK)”:http://www.greaterkashmir.com/full_story.asp?Date=28_8_2008&ItemID=10&Cat=1. This number definitely goes higher when we take in account the death in the hospital due to bullet injuries.

Even hospitals are not spared. Residents of Handwara township in north Kashmir on Wednesday said that Indian paramilitary CRPF troopers and policemen of firing tear smoke canisters inside an operation theatre in the Sub district hospital Handwara, when a civilian hit by a bullet of the soldiers during a demonstration was being operated upon by doctors. (read the full story on GK).

Strong resentment is brewing against the troopers of paramilitary CRPF for allegedly harassing the inhabitants and damaging their property during curfew in Habba Kadal locality of Shehr-e-Khas locality on Wednesday. (read the full story on GK).

And during all this time fourth pillar of democracy has been simply shattered to un-assemblable peices (read the full story on GK).

Yesterday, United Nations Office of the High Commissioner for Human Rights (UN-OHCHR) voiced its concern about the recent protests in Kashmir that have led to civilian casualties and restrictions to the right to freedom of assembly and expression.

OHCHR Spokesman in Genev, noted in a statement:

“OHCHR calls on the Indian authorities and in particular security forces to respect the right to freedom of assembly and expression, and comply with international human rights principles in controlling the demonstrators,”
“The use of force should be proportionate to the threat posed and firearms must only be used in dispersing a violent assembly to protect individuals against an imminent threat of death or serious injury” (read the full story on GK).

Mr. Spokesman has forgotten that Indian Forces are here with an open license to kill us from the GoI. These murders are being awarded with promotion, medals and honors. Kill one Kashmir, Get a medal, Kill two, get a promotion, Kill 10 get Param Veer Chakra.

I say, We give a damn shit! Kill One or Kill All. We will still fight and struggle and not give up our way. The revolution is here. You can’t escape it.

From Penci Project webiste:

“The Pencil Project’s unique mission is to build a free and open source tool for making diagrams and GUI prototyping that everyone can use.”

Main features include:

• Built-in stencils for diagraming and prototyping

• Multi-page document with background page

• On-screen text editing with rich-text supports

• PNG rasterizing

• Installing user-defined stencils

• Standard drawing operations: aligning, z-ordering, scaling, rotating…

• Cross-platforms

• Adding external objects You got to use it to believe it. Check out some real cool screen shots. A user guide is OK as of now, but there are a couple of screencasts that are a good start.

So far its a great work. Congrats and thanks to guys at Evolus.

It could definitely use some good features like support for SVG, OmniGarffle and Visio files etc.

Check this Video and read more in Aza’s blog post