Source: CareerProNews

Just as miners of the past struck out in search of valuable ore, a new brand of miners is digging and searching today. But this time around, the pack mule is a laptop computer and the pickaxe is a computer mouse. The new prospectors are data miners.

The purpose of data mining is to search for and find valuable information in a mountain of computer data. That data may be in a company’s computer system or its website. Companies then use the information found by the data miners for things like following trends or making strategies.

Michael Berry is the founder and principal manager of a data mining company. He says that he was in the right place at the right time to be involved in data mining, almost from the very start.

“We certainly didn’t invent any of the ideas, but in the ’90s they were just starting to get used a lot. And my partner that I started the company with and I had written a book on data mining techniques which was — at that time — one of the few out,” Berry says. “So we started getting a lot of inquiries.”

Those inquiries led to requests for consulting services. “And after enough people started asking, we decided that maybe the answer ought to be yes,” he says.

Berry says data mining can be broken down into two basic categories. “In one, you don’t have a specific goal, but you’re interested in finding patterns that may exist in the data. And in another, you have a goal in mind — like something you’d like to try to explain.”

One way a company can use the patterns that show up in the data is by getting to know its customers or clients better.

“Anyone who buys books from Amazon.com more than once recognizes that. The second time you come back, you’re offered some things that are similar to what you bought the first time,” says Berry. “Behind those so-called ‘recommendation engines’ or ‘personalization engines’ is data mining. So the Web is a consumer of data mining.”

Another way companies can benefit from data miners is by studying the trends of certain types of people — like people who may be a credit risk. “We have examples of people who have always paid their bills, and examples of people who have failed to pay their bills,” says Berry.

“There are techniques for trying to discriminate between the two. Trying to say, ‘What is it about the ones that didn’t pay that is different from the ones that did pay?’ This will…help you come up with a scoring system. When someone applies for credit…you see them as more or less risky because of how similar — or dissimilar — they are to the people who have not paid in the past.”

Daniel Silver is also a pioneer in the field of data mining. He runs a company that offers consultation and education in data mining. Silver says it’s interesting to see how far data mining has come in such a short time.

Banks and phone companies started the ball rolling, Silver says. Then smaller companies picked up on what the larger firms were doing, especially in regards to data mining done on the Internet.

Data mining is still relatively new in North America. Both Silver and Berry say it’s hard to know how many people are actually employed in the field at this time.

“It’s going to be really hard to come up with that number, because very few people are called that, even though that may be a large part of their job,” Berry says.

“They might be called database marketing analysts, credit risk analysts or direct-mail marketing managers. And yet a lot of what all of these people do is examine data and try to come up with models that will help them do those jobs better.”

Silver and Berry agree that job opportunities in the field of data mining will continue to rise as more companies look for ways to get a competitive edge.

Silver says that teaching others about data mining can be even more lucrative and popular than doing it. He teaches courses at both the university and the business levels.

Looking down the road a few years, Berry sees more and more companies hiring data mining services. He also expects businesses to give data miners a more prominent position in the company.

“[They will] be better integrated with other activities inside the company, not an isolated analytical exercise in the corner,” he says. “[They will] become part of the normal way of doing everyday business.”

Silver says that colleges and universities have also recognized the trend and have reacted. For example, schools such as Berkeley and the University of California at Los Angeles are now offering programs in computational finance.

Berry says that high school students can do a few things today to prepare for careers as data miners. One of the most important steps they can take is to get good grades in math.

“They should continue to take their math courses, because math is at the heart of it,” he says.

“And if they have a chance, [they should take] probability and statistics. That’s the part of math that’s used a lot in this kind of work. Computer science is also going to be valuable. All of this kind of work ends up being done on computers.”

Net Sites

The Data Mine
This site offers a wide variety of data mining information
http://www.the-data-mine.com/

Data Miners
One of the first businesses in the U.S. to take advantage of the demand for data miners
http://www.data-miners.com/

Elementary Concepts in Statistics
You’ll need to know this stuff to do data mining
http://www.statsoft.com/textbook/esc.html

Knowledge Discovery and Data Mining
Tips and tools used in data mining
http://www.kdd.org/