View the presentation slides  (5.5 Mb)

Abstract

Computing has become an integral part of all scientific research from data acquisition, to data management, analysis, collaboration and publications. A challenge becomes how to incorporate advances of software and technology into systems and workflows without major disruptions to researchers. To make eScience or eResearch mainstream, one of the “last mile” problems, making High Performance Computing and Data Analysis available to all researchers needs to be overcome. One approach to this is to utilize high performance computing, data management and analysis solutions developed for industry and apply those solutions to scientific challenges.

While databases are used quite extensively in businesses for managing data, the use of in scientific domains has been slow in adoption. Also the use of mainstream data mining and analysis tools has proven itself in industry could benefit scientific research. A couple examples of the use of databases and data mining tools/analysis in specific scientific domains (protein folding, astronomy, and hydrology) will be examined.

Finally the GrayWulf cluster concept will be presented – GrayWulf brings together HPC Beowulf clusters and SQL databases to enable computations and data management to be more tightly coupled. This combination not only brings the data closer to the computation, but also leverages the best of both technologies for the scientific insight.

About the speaker

Daniel Fay is the Director of External Research for Earth, Energy, and Environment engagements for Microsoft Research, where he works with academic research projects focused on utilizing computing technologies to aid in scientific and engineering research. Dan was previously the manager of eScience Program in Microsoft Research as well as program manager for Project 7, a unique project that brought together a number of language researchers to examine and provide feedback on the Common Language Runtime and its multi language support. Before joining Microsoft in 1992, Dan was a senior software engineer with Digital Equipment Corporation working on OSF/1 and embedded network devices. Dan is a graduate of Northeastern University in Electrical Engineering.