Thursday, October 02, 2008

Music Video Without Cameras

While browsing Google's 10th birhtday site I ran across a cool video that cought my eye. It gave a behind the scenes look at the creation of Radiohead's "House of Cards" music video.


It turns out the entire video was shot without using a single camera or strobe. Instead, they used two experimental technologies, Geometric Informatics and Velodyne LIDAR that capture 3D models in real time. The former was used to create the close-up shots, while the latter was used for the landscapes.

Google also published a data viewer that lets you manipulate the 3D model to get a sense of the data captured, its properties, and size. The really cool part is that you can move Thom Yorke's head around while the video is playing! As a data visualization geek, this sort of stuff really gets me going.

Read more...

Saturday, September 06, 2008

Microsoft Thinks Different

I just saw the first Microsoft Windows ad featuring Bill Gates and Jerry Seinfeld. This ad campaign is Microsoft's answer to Apple's Mac vs. PC ads which have been going unchallenged for over two years.


I don't know if the ad is working or not, it's probably too soon to tell, but it's definitely been a long while since a Microsoft ad campaign made such a buzz. There is a lot of talk about it in the blogosphere and even in mainstream media.

According to the Wall Street Journal, Microsoft is spending $300M on this ad campaign (Jerry's cut is $10M), so it better work. After the annoying and the what-are-you-trying-to-prove? that-everyone-thinks-Vista-sucks! campaign a.k.a.  "Mojave Experiment" the expectations are running high.

It's ironic that when you search for "Seinfeld ad" in YouTube, the first result is a Mac ad :-)


Read more...

Wednesday, September 03, 2008

The APM, Anonymous Methods and Lambda Expressions

In this post I would like to continue from where I left of in my previous post about the Asynchronous Programming Model, and show how the APM could be more concise using Anonymous Methods and Lambda Expressions.

Using "classic" delegates the code to implement the APM looks like this (as shown in greater details in my previous post):

public void ClassicAsync(int num)
{
DateTime start = DateTime.Now;
Console.WriteLine("Starting async calculation at: {0}", start);

PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;
del.BeginInvoke(num, CalcCompleted, start);
}

private void CalcCompleted(IAsyncResult result)
{
DateTime start = (DateTime)result.AsyncState;

Func<int, int> del = (Func<int, int>)((AsyncResult)result).AsyncDelegate;
int prime = del.EndInvoke(result);

DateTime end = DateTime.Now;
TimeSpan length = end.Subtract(start);
Console.WriteLine("Completed async calculation at: {0}", end);
Console.WriteLine("Async calculation took {0:F} seconds.", length.TotalSeconds);
}


With an anonymous method we can write the callback method inline like this:
public void AnonymousAsync(int num)
{
DateTime start = DateTime.Now;
Console.WriteLine("Starting anonymous async calculation at: {0}", start);

PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;

del.BeginInvoke(num, delegate(IAsyncResult result)
{
int prime = del.EndInvoke(result);
Console.WriteLine(prime);

DateTime end = DateTime.Now;
TimeSpan length = end.Subtract(start);
Console.WriteLine("Completed anonymous async calculation at: {0}", end);
Console.WriteLine("Async calculation took {0:F} seconds.", length.TotalSeconds);
}, null);
}


And, finally, using a lambda expression the implementation is even more concise:
public void LambdaAsync(int num)
{
DateTime start = DateTime.Now;
Console.WriteLine("Starting lambda async calculation at: {0}", start);

PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;
del.BeginInvoke(num, result =>
{
int prime = del.EndInvoke(result);
Console.WriteLine(prime);

DateTime end = DateTime.Now;
TimeSpan length = end.Subtract(start);
Console.WriteLine("Completed lambda async calculation at: {0}", end);
Console.WriteLine("Async calculation took {0:F} seconds.", length.TotalSeconds);
}, null);
}

Read more...

Monday, September 01, 2008

Using Delegates to Implement APM

In my previous post I described how delegates have evolved over the various versions of the .NET framework. In this post I want to elaborate and describe how to use delegates to implement asynchronous programming.

Many of today's PCs have multiple processors/cores in them, and there's a definite trend among processor makers to gradually increase that number in future models. In order to utilize the increasing number of cores, computer programs need to parallelize their execution, and assign processor intensive tasks to dedicated threads. Writing robust multi-threaded software is not easy. Threads need to be synchronized, data locked, and dead-locks are difficult to avoid. This increases the challenge for developers who need to write robust and efficient multi-threaded code.

To help programmers out, Microsoft introduced the Asynchronous Programming Model which simplifies multi-threaded programming. They have implemented it themselves in many classes throughout the .NET framework such as the FileStream, Socket, WebRequest, SqlCommand, etc. All of these classes offer asynchronous method calls along-side the standard synchronous versions. The async method name always starts with BeginXxx, and offers a corresponding EndXxx. For example, the SqlCommand class offers an ExecuteReader method, plus an async version of the method - BeginExecuteReader. To comply with the APM it also offers an EndExecuteReader method.

All delegates implement this pattern out-of-the-box. The all offer the standard Invoke method to execute the target method, and also offer the BeginInvoke and EndInvoke methods to execute the target method asynchronously.

This is the main reasons I love delegates so much.

Let's get down to it. One of the complexities of multi-threaded programming is figuring out when an async process completed its work. This is where the APM offers a great deal of help. There are three techniques to find out when a BeginInvoke is done: wait, poll and callback. I'll use a sample to describe the three techniques.

class PrimeCalc
{
public int GetNextPrime(int num)
{
int p = num + 1;
while(true)
{
if(IsPrime(p))
{
break;
}
p++;
}
return p;
}
}


The PrimeCalc class has a GetNextPrime method that calculates the next prime number after the number passed in to the method. The class uses some internal methods to do the calculation (I'll add the entire source code at the end of the post) and returns the next prime number it finds. We'll use this method to simulate a compute-intensive task.

Since our target method takes an int as a parameter and returns an int as a return value, we could use the Func< as our delegate:

PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;


Wait Until Completed Technique



public void WaitUntilComplete(int num)
{
PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;
IAsyncResult result = del.BeginInvoke(num, null, null);

// Do some other work here

// Suspend this thread until the async operation completes
int prime = del.EndInvoke(result);
}


This is not a very efficient way of using the APM. If the compute-intensive task takes longer than the "other" work that is being done in the mean time, the thread will be suspended and wait for the delegate to return.

Polling Technique



public void Poll(int num)
{
PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;
IAsyncResult result = del.BeginInvoke(num, null, null);

while (!result.IsCompleted)
{
// Do some other work here
}

// Get the result from the async operation
int prime = del.EndInvoke(result);
}


This option is also not very efficient because the while loop that runs until the async operation completes would consumes CPU resources even after the "other" work that needs to be done is complete.

Callback Technique



This is by far my favorite rendezvous technique of all, it is a bit more complex to implement, but offers greater flexibility and control, and is more efficient in terms of resource management.

public void Callback(int num)
{
DateTime start = DateTime.Now;
Console.WriteLine("Starting async calculation at: {0}", start);

PrimeCalc calc = new PrimeCalc();
Func<int, int> del = calc.GetNextPrime;
del.BeginInvoke(num, CalcCompleted, start);
}

private void CalcCompleted(IAsyncResult result)
{
DateTime start = (DateTime)result.AsyncState;

Func<int, int> del = (Func<int, int>)((AsyncResult)result).AsyncDelegate;
int prime = del.EndInvoke(result);

DateTime end = DateTime.Now;
TimeSpan length = end.Subtract(start);
Console.WriteLine("Completed async calculation at: {0}", end);
Console.WriteLine("Async calculation took {0:F} seconds.", length.TotalSeconds);
}


The basic idea here is that the delegate calls the callback method when it completes. The signature of the callback method should match the AsyncCallback delegate signature:

private void CalcCompleted(IAsyncResult result)


This time, when we call the BeginInvoke method on our delegate, we pass in the callback method as the second parameter, and the third parameter could be any state object we would like to pass in to the EndInvoke method. In this sample I passed in the start DateTime of the async opertaion.

In the callback method you should get a reference to the delegate instance (by casting the result param to the appropriate type), and use it to call EndInvoke. It is important to call EndInvoke even if your method does not return a value. This prevents reasource leaks.

You get a reference to the state object you passed in the BeginInvoke 3rd parameter by casting the result's AsyncState property to the appropriate type.

Here is the entire PrimeCalc class source code:


namespace AsyncProgrammingModel
{
class PrimeCalc
{
private List<int> primes;

public PrimeCalc()
{
primes = new ListList<int>();
primes.Add(2);
primes.Add(3);
}

public int GetNextPrime(int num)
{

int p = num + 1;
while (true)
{
// Create a list of all prime numbers up to p.
AddPrimesToList(p);
if (IsPrime(p))
{
break;
}
p++;
}
return p;
}

private void AddPrimesToList(int numberToTest)
{
int n = primes[primes.Count - 1] + 2;
while (n < numberToTest)
{
if (IsPrime(n))
{
primes.Add(n);
}

// Skip even numbers.
n += 2;
}
}

private bool IsPrime(int n)
{
bool foundDivisor = false;
bool exceedsSquareRoot = false;

int i = 0;
int divisor = 0;

// Stop the search if:
// there are no more primes in the list,
// there is a divisor of n in the list, or
// there is a prime that is larger than the square root of n.
while ((i < primes.Count) && !foundDivisor && !exceedsSquareRoot)
{
// The divisor variable will be the smallest
// prime number not yet tried.
divisor = primes[i++];

// Determine whether the divisor is greater than the square root of n.
if (divisor * divisor > n)
{
exceedsSquareRoot = true;
}
// Determine whether the divisor is a factor of n.
else if (n % divisor == 0)
{
foundDivisor = true;
}
}

return !foundDivisor;
}
}
}

Read more...

Friday, August 29, 2008

The Evolution of Delegates

In this post I'll discuss how delegates have evolved along with the evolution of .NET and C#. Specifically, how the introduction of anonymous methods and lambda expression helped make delegates more concise and useful.

This post is the first in a series of "Back-to-Basic" posts where I'll discuss basic .NET concepts and show how they have progressed with each new release of the .NET Framework.

I love delegates. If I had to come up with a list of my top 5 .NET features, delegates would definitely be high up on the list.

A Delegate, as defined by MSDN is:

"A type that defines a method signature, and can be associated with any method with a compatible signature. You can invoke (or call) the method through the delegate. Delegates are used to pass methods as arguments to other methods."

Delegates are first class citizens in the .NET framework and the C# language, and one of the fundamental types in the framework. They are used throughout the framework and enable important features such as events and asynchronous method calls.

In the old days, if we wanted a delegate that takes one parameter and returns void we had to declare one like this:

public delegate void DoWorkDelegate(string data);

Then we had to implement a method that conforms to the signature of the delegate:

private void DoSomeWork(string s)
{
// Do some work here
Console.WriteLine(s);
}

And finally, instantiate the delegate in our code:

DoWorkDelegate del = new DoWorkDelegate(DoSomeWork);

In .NET 2.0 we could use a simplified syntax to instantiate the delegate:

DoWorkDelegate del = DoSomeWork;

The introduction of Generics in .NET 2.0 introduces a generic delegate Action. Action encapsulates a delegate that returns void and accepts 0 to 4 parameters (there are actually 5 generic Action types defined: Action, Action<T>, Action<T1, T2>, Action<T1, T2, T3>, Action<T1, T2, T3, T4>). So in the above example, we could have saved us the trouble of defining our own custom delegate "DoWorkDelegate" and instead used this:

Action<string> del = DoSomeWork;

.NET 2.0 also introduced the concept of Anonymous Methods. This lets us avoid defining the "DoSomeWrok" method altogether and write it inline with the definition of the delegate:

Action<string> del = delegate(string s)
{
Console.WriteLine(s);
};

Finally, C# 3.0 introduced Lambda Expressions, which allow us to define the method inline like this:

Action<string> del = s =>
{
Console.WriteLine(s);
};

One more delegate worth mentioning is the Func delegate. Func was introduced in .NET 3.5 and just like its older brother, the Action, it encapsulates a delegate, but unlike Action Func encapsulates a delegate that takes 0 to 4 and returns a value. For example:

// Using Func we can replace this:
delegate int DoWorkDelegate(string data);

// with this:
Func<string, int> del;

In all the above samples (except for the last one) we end up with a "del" object. We could then use the "del" instance anywhere in our code to execute the "DoSomeWork" method. We could even pass "del" as an argument to other methods or objects that could also execute the same method:

// This is the verbose way:
del.Invoke("Do some work");

// This is a bit more concise
del("Do some work");

// This executes the method asynchronously
del.BeginInvoke("Do some work", null, null);

Next time on SapienCoder I will describe how to use delegates to implement Asynchronous programming patterns.

Read more...

Tuesday, August 26, 2008

Will reCAPTCHA Save Humanity?

Probably not, but it will make us more productive during the tedious process of proving our humanhood to some random web server.

Anyone who's using the internet these days for anything more involved than reading news knows those annoying sets of garbled characters that appear at the end of checkout and registration pages.

captcha

Those are called CAPTCHAs (an acronym for "completely automated public Turing test to tell computers and humans apart") and they are designed to test that we, the users of the web site are in fact human. Now, I don't disagree with the motivation behind CAPTCHAs. Today's Web is infested with crawling bots and other malicious agents running around causing all kinds of mayhem. But, it just seems like a terrible waste of time for something that aught to be straight forward (how many times have you mistyped a CAPTCHA and had to enter it over and over again until the server acknowledged your humanhood?).

Luis von Ahn, the inventor of the CAPTCHA, said in an interview recently that by his estimates people spend an average of 10 seconds solving one of those CAPTCHA puzzles. Multiply that by the number of CAPTCHAs solved daily (aprox. 200 million) and you come to the staggering number of approximately 500,000 hours per day world wide. Astonishing when you think about it in these terms.

That's exactly why he came up with the brilliant idea of the reCAPTCHA.

There is a growing number of libraries and archives who are working on digitizing their entire collections. The process involves scanning the printed document (book, newspaper, magazine, historical document, etc.) to an image file, and then running a software called OCR (for Optical Character Recognition) that tries to recognize the words in the scanned image and turns them into a searchable text document. It turns out that the OCR software can't "read" every document with 100% accuracy. That's where the power of crowd-sourcing comes in. recaptcha ReCAPTCHA uses the words that computers can't decipher with OCR software (therefore can't be "read" by malicious robots) and displays them to humans. Each reCAPTCHA actually consists of two words, one was successfully recognized by the computer and the other wasn't. Each image is also shown to several people to verify the accuracy of the translation. If they agree on the translation the transcription is considered accurate and will be added to the text it originally came from. Currently more than 40,000 web sites world wide are using reCAPTCHA technology including some you might have heard of like Ticketmaster, Facebook and CraigsList. There are plug-ins for WordPress, Joomla, Drupal and many other popular web applications, as well as APIs for PHP, ASP.NET, Java, Perl, Ruby etc. The implementation is simple and there are lots of resources available for site developers.

Next time you run into those squiggly characters when commenting on a blog, or signing up for an online email account, make sure your time is not wasted on a standard CAPTCHA text. Let the site owners know your time could be spent on saving humanity, or at least it's written word.

Read more...

Thursday, August 07, 2008

The Olympics on Your PC


When the Olympic flame is fired up in Beijing tomorrow, and the Games begin, many people will realize that they can't watch their favorite athlete or event because of the time zone difference with China. Well this year is the first time ever that most of the events will also be available in high quality HD directly on your PC at any time you like (yes, even at work when your boss is not looking).

NBC had partnered with Microsoft
to deliver the 2008 Beijing Summer Games over the internet. At first NBC considered going with Adobe Flash which is considered the industry standard for Web video, but Microsoft was able to convince NBC that their Silverlight technology will deliver better quality to the end user. They plan on streaming most of the events live, and also make them available as on-demand content for your viewing pleasure any time you like.

Thanks to Silverlight, NBC can offer a unique viewing experience not even available on high end TVs. For example, they will offer a picture-in-picture view of two simultaneous events, and, for the real sports junkies, a "control room" mode with four events streaming side by side on the screen.

I can't imagine the complexity involved in delivering such a massive amount of data to so many users in real-time. This drawing (taken form an interesting News.com article) tries to explain it:



It is interesting to see how this "experiment" turns out, and if the masses are ready for Web TV (and if the technology stack is there too).

Don't forget to check out the NBC Beijing 2008 site at NBC Olympics.

Read more...

  © Blogger template Blogger Theme II by Ourblogtemplates.com 2008

Back to TOP