Now let’s analyze OkCupid.com it is a famous web site. On 2-Feb-2011 Match.com had acquired OkCupid for $50 million in cash. Many users expressed distress, worrying that they would now have to pay for OkCupid’s free dating services. Well, co-founder and CEO Sam Yagan, says that this isn’t the case. Now all users should be happy? Should I? No I am not a user? My relation to OkCupid started with their Okcupid Blog. They have funny talented statisticians and they like to run crazy analysis over their large set of data. They have interesting tests as well. İf you have not checked them out, you are decreasing your chance of finding the right one by 70%.

Remember we have talked about the Netflix Pragmatic Chaos algorithm. In this case the algorithm is not known or disclosed but thanks to their mathematicians that they are sharing interesting findings in their data set. OkCupid has the power of running millions of questions over thousands of their users in fairly small fraction of time. And they can easily come up with statistically very significant correlations. As we all know whaen the sample size increase the results get more and more significant.

1- If you are a okcupid user your best match is determined by an ALGORITHYM

2- How attractive you are depends on your photo!

This is how they are rated? For Woman an advice, A very significant advice:

“The more men disagree about a woman’s looks, the more they like her.”

The bar at the bottom represents the like count of those images in a scale of 1 to 5. Which one do you think got the most messages?

Ok final words: From OkCupid

“…But our advice can apply to anyone. Browsing OkCupid, I see so many photos that are clearly designed to minimize some supposedly unattractive trait—the close-cropped picture of a person who’s probably overweight is the classic example. We now have mathematical evidence that minimizing your “flaws” is the opposite of what you should do. If you’re a little chubby, play it up. If you have a big nose, play it up. If you have a weird snaggletooth, play it up: statistically, the guys who don’t like it can only help you, and the ones who do like it will be all the more excited.”

On the other side of the equation we have companies like Epagogix. In their web site they define their business as “…Epagogix helps studios to identify, enhance and deliver on-screen success, and guides investors in the creation of winning film-related investments. Epagogix works confidentially with the senior management of major film studios and large independents, assisting with the selection and development of scripts, helping to transform scripts with low Box Office revenue potential into properties that can be profitably produced and distributed, and identifying scripts with a low probability of commercial success….”

That means they come with movies that we will LOVE.

On one side of the equation we have netflix (telling you what to watch) = on the other side we have Epagogix (telling the producers what to produce)

Where is myself in the equation.

List of mining techniques used by NetFlix

After a few days

From http://www.michaeleisen.org/blog/?p=358 

Michael Eisen’ Blog: “….But two questions remained. Why were they doing this, and how long would it go on before they noticed? As I amusedly watched the price rise every day, I learned that Amazon retailers are increasingly using algorithmic pricing (something Amazon itself does on a large scale), with a number of companies offering pricing algorithms/services to retailers. Both profnath and bordeebook were clearly using automatic pricing – employing algorithms that didn’t have a built-in sanity check on the prices they produced. But the two retailers were clearly employing different strategies….”

A must read for every Data Scientist !

Stock prices can go down as well as up. Never in financial history has this adage been more apt than on 6 May 2010. Then, the so-called “Flash Crash” sent shocks waves through global equity markets. The Dow Jones experienced its largest ever intraday point fall, losing $1 trillion of market value in the space of half an hour. History is full of such fat-tailed falls in stocks. Was this just another to add to the list, perhaps compressed into a smaller time window?
No. This one was different. For a time, equity prices of some of the world’s biggest companies were in freefall. They appeared to be in a race to zero. Peak to trough, Accenture shares fell by over 99%, from $40 to $0.01. At precisely the same time, shares in Sotheby’s rose three thousand-fold, from $34 to $99,999.99. These tails were not just fatter and faster. They wagged up as well as down.

This is the Flash Crash:

we remain unsure quite what caused the Flash Crash or whether it could recur.

You can read more at: The race to zero In a speech at the International Economic Association Sixteenth World Congress in Beijing, Andrew Haldane – Executive Director for Financial Stability and member of the interim Financial Policy Committee – outlines how dramatic shifts in the structure and speed of trading have increased abnormalities in the pricing of securities.

Nanex is an interesting company.

In their site they say”NxCore (pronounced n’core) is a high-performance, real-time streaming datafeed (ticker plant) that brings the whole market to your workstation or desktop computer. NxCore excels in delivering and databasing all the quotes and trades transmitted by the exchanges, even in the hyper-active U.S. Option market (OPRA) which now transmits over 4,500,000 quotes per second, and 8 billion quotes per trading day.”

Imagine if you were using that much of information and enough mathematicians to come up with “clever” algorithms for trading. CLEVER? İn nanex web site they disclose some interesting research as well. (That made them very special for my work) NANEX Research.  The news in the list is amazing. They publish about the CLEVER algorithms that they notice. You can read about DELL, NVIDIA, etc stories. Those are called BUGS in software development. They are the bugs running through financial markets. Unfortunately there is no way we can see them. Except web sites like NANEX research.

As performance of CPU’s increase the High Frequency Trading is also increasing. the Graph above clearly shows this trend. Where will it stop is an other question?

Look at those CLEVER trading out comes!

Blue Zinger

Detox

Knife

So CLEVER.

By Yavir Zur http://vimeo.com/user2924863

In this presentation I tried to explain why big data or extreme data problems are so trendy. There is a specific reason for this. For those who attended the event I hope it is clear. For those who have not attended you need to walk over the presentation to have an idea.

Our current education system is engineered during the industrial revolution and have not seen radical changes ever since. With the emerging mobile technologies I do believe that the learning experience could dramatically change. The main problem is having a batch of students receiving the same educational curriculum. 

1- Unfortunately the speed and the interest level of all students vary dramatically and without technological help it is not possible to engineer unique learning experience for the individual students.

2- In today’s learning environment we must understand why we all like playing games and hate having boring lessons. We need to engineer and make use of the 47 game mechanics during a lesson. It is possible to embed learning experience to lessons and embed learning objects to games. Both are possible.

Since we are all clear about the roadmap we need to establish a technical landscape that would enable the vision. But how? Very simple all we need to do is to mimic what all those mega game exchanges are doing. We should benchmark our education system against iTunes, BlackBerry Apps market, Sony PlayStation Network etc. I am sure we can engineer a better environment.

On the otherside of the equation we will have the games itself. We also need to re-engineer the games. Garry Kasparov (the most powerful chess grand master that we know) is designing a learning experience for students between 6-9 years of age. It is important to know that stating chess at the age of 6-9 can improve our kids IQ, but after 9 years old the marginal benefit from learning or playing chess decreases. You should also keep this in mind. On 9th of December 2011 we have presented this idea to Minister of Education Omer Dincer and his team responsible for Fatih project.

I would just answer those questions if you ask me to?

This is a description of findingrecords.gif. The image contrasts the paths of an application request to a disk-based RDBMS and TimesTen.

In the disk-based RDBMS, the application uses a SQL statement to initiate a request for a record. The statement is passed over an IPC connection to a query optimizer, which uses table numbers and page number to locates the pointer to a page in the buffer pool by using hashing and linear search methods. The record is copied into a private buffer and then passed across an IPC connection to the application.

In Timesten, the application also uses a SQL statement to initiate a request for a record. The statement is passed by direct link to a query optimizer, which determines the memory address of the record. The database has already been loaded from checkpoint files on disk into memory. The record is copied into the application buffer and returned to the application.

55 minutes to 2 seconds 1650x Sales Reporting in a retailer in Turkey. With SAP HANA a report which lasted 55 minutes were reported over the in-memory technology of SAP in 2 Seconds. That means a 1650x improvement. See it for yourself

Oscar Leonard Carl Pistorius (born 22 November 1986) is a South African sprint runner. Known as the “Blade Runner” and “the fastest man on no legs”, Pistorius, who has a double amputation, is the world record holder in the 100, 200 and 400 metres (sport class T44) events and runs with the aid of Cheetah Flex-Foot carbon fibretranstibial artificial limbs by Ossur. In 2007 Pistorius took part in his first international competitions for able-bodied athletes. However, his artificial lower legs, while enabling him to compete, have generated claims that he has an unfair advantage over able-bodied runners. The same year, the International Association of Athletics Federations (IAAF) amended its competition rules to ban the use of “any technical device that incorporates springs, wheels or any other element that provides a user with an advantage over another athlete not using such a device”. It claimed that the amendment was not specifically aimed at Pistorius. After monitoring his track performances and carrying out tests, scientists took the view that Pistorius enjoyed considerable advantages over athletes without prosthetic limbs. On the strength of these findings, on 14 January 2008 the IAAF ruled him ineligible for competitions conducted under its rules, including the 2008 Summer Olympics. This decision was reversed by the Court of Arbitration for Sport on 16 May 2008, the Court ruling overall there was no evidence that Pistorius had any net advantage over able-bodied athletes.

Although eligible to compete in the 2008 Summer Olympic Games in Beijing, Pistorius did not qualify for the South African team. Despite achieving third place and a personal best time of 46.25 seconds in the 400 metres in Lucerne, Switzerland, on 16 July 2008, this was short of the Olympic qualification time of 45.55 seconds. He was also not selected by the South African Olympic Committee for the 4 x 400 metres relay team as there were four other runners who had achieved better times.

What if he wins. Would it have the same impact like Kasparov? Remember the time he was beaten by a Computer.

No this is the New Era: Human Machine Integration