Performance Evaluation: Origins and Directions (2000)

Performanceevaluationhas beena discipline ofcomputersciencefor somethirty years. To us, it seemed to be time to take stock of what we - the performance evaluation community - were doing. Towards this end, we decided to organize a workshoponPerformanceEvaluationofComputer SystemsandCommunication Networks, which was held at the international conference and research center for computer science, Schlo Dagstuhl, Germany, September 15-19, 1997. The participantsdiscussed, amongotherthings, thefollowingfundamentalquestions: What are the scienti c contributions of performance evaluation? What is its relevance in industry and business? What is its standing in academia? Where is the eld headed? What are its success stories and failures? What are its current burning questions? During this workshop, working groups focused on areas like performance ev- uation techniques and tools, communication networks, computer architecture, computer resource management, as well as performance of software systems (see http: //www. ani. univie. ac. at/dagstuhl97/). The participants summarized the past of performance evaluation and projected future trends and needs in the eld. It was identi ed that - as in many other sciences - at the beginning therewastheobservationofthebehaviorofsystems, generallybymeasurements, followed by the development of theories to explain the observed behavior. - pecially in system modeling, based on these theories, methodologies have been developed for behavior prediction. At that stage, measurement changed its role from pure phenomenological observation to model-driven parameter estimation. Basedonaseriesofhighlysuccessfulcasestudies, toolmethodologyimplemented in versatilesoftwarepackageshas been developed to make the theoretical results amenable to practitioners.