数据库基准测试（Database Benchmarking） 说明

1   TPC概述

对数据库和服务器进行压力测试或做对照的时候，都是常常会使用HammerDB 或 Benchmark之类的工具，而且最经常使用TPCC的标准进行测试。

    这里说的TPCC，前3个字母是：TPC 是Transaction Processing Council（事务处理委员会），其提供了量化的方法和标准。

    所以我们说的TPCC就是其中1种测试方法。  

    TPC的官网是：http://www.tpc.org/

从图上可以看出，除TPC-C 以外，还有很多其他的方法，包括废弃的TPC-A,TPC-B,TPC-D,TPC-R,TPC-W。

关于不同的测试方法的区分可以直接点击官网的说明。

2   Oracle 官网对Benchmarking的说明

下文直接引自Oracle 的网站，原文地址已不存在，文章讲了database benchmarking的缘由、标准，和几款丈量工具软件。

2.1             Reasons for Benchmarking

Benchmarks are performed for variousreasons. However, benchmarks are primarily used:

* To compare different hardware configurations

Benchmarks can be used to compare the relative performance ofdifferent hardware running the same application. This isgenerally used to directly compare hardware configurations between two hardwarevendors.

* To compare different database vendor software

By running the same benchmark usingdifferent database software on the same machine, one can easily compare betweendifferent database vendors. This is generally usedto make a price/performance decision between vendors such asOracle, Microsoft, IBM, etc.

* To compare different database software releases

Similar to the above, one can use different versions of the samevendor's database software to compare the one they want to use or checkfor performance regressions due to upgrades (i.e. 10g vs. 11g).

2.2             Real World Benchmarks

While there areseveral industry standard benchmarks, most of the time, they don'treflect a company's actual workload. As such, you will often findcustom benchmarks being performed. The goals of a real-world benchmark are thesame reasons mentioned above.

2.3             Industry Standard DatabaseBenchmarks

Industrystandard benchmarks are generally used by businesses to compare differenthardware and software system performance forpurchase-related reasons. While industry standard benchmarks are modelled afterreal-world workloads, they rarely reflect a company's realworkload. However, it is a quick way to compare the performance of varioushardware and software combinations operating in a well-defined scenario. Themajor industry standard benchmarks are listed below.

2.4             Transaction Processing Council(TPC)

The TPC is anon-profit corporation which supports a consortium of hardware and databasesoftware vendors devoted to defining transaction processing anddatabase-related benchmarks. The primary goal behind TPC benchmarks is thedefinition of functional requirements which can be run on any database,regardless of the hardware or operating system. This allows vendors toimplement their own benchmark kits in order to satisfythe functional requirements. Similarly, after having publicly submitted proofthat a benchmark was conducted according to the specification, end-users havemore of an assurance that what they are presented with is avalid, apples-to-apples comparison.

2.4.1 TPC-B

The TPC-B benchmark stresses databases and is characterizedby significant disk input/output, moderate system, application execution time,and transaction integrity.  Thisbenchmark targets database management systems (DBMS) batch applications, andback-end database servers.  TPC-B is notan OLTP benchmark.

2.4.2 TPC Benchmark C (TPC-C)

The TPC-C simulates an order-entryenvironment where apopulation of terminal operators executes transactions against a database. Thebenchmark is comprised of transactions which include entering and deliveringorders, recording payments, checking the status of orders, and monitoring thelevel of stock at the warehouses. The most frequent transaction consists ofentering a new order which, on average, iscomprised of ten different items. Each warehouse tries to maintain stock for the 100,000 itemsin the Company's catalog and fill orders from thatstock. The performance metric reported by TPC-C measures the number of ordersthat can be fully processed per minute and is expressed in tpm-C. The TPC-Cwill soon be deprecated in favor of the TPC-E.

2.4.3 TPC-D

The TPC-D benchmark represents a broad range ofdecision support (DS) applications that require complex, and long runningqueries against large complex data structures. Real-world business questions were written against this model, andresulted in 17 complex queries.

2.4.4 TPC Benchmark E (TPC-E)

The TPC-E benchmark simulates the OLTP workload of a brokeragefirm. The focus of the benchmark is the centraldatabase that executes transactions related to the firm’s customer accounts.The TPC-E metric is given intransactions per second (tps). It specifically refers to the number ofTrade-Result transactions the server can sustain over a period of time.

2.4.5 TPC Benchmark H (TPC-H)

TheTPC-H is adecision support benchmark which consists of several business-oriented ad-hocqueries and concurrent data modifications. This benchmark illustrates decisionsupport systems that examine large volumes of data, execute queries with a highdegree of complexity, and give answers to critical business questions.

The performancemetric reported by TPC-H is called the TPC-H Composite Query-per-HourPerformance Metric (QphH@Size), and reflects multiple aspects of the capabilityof the system to process queries.

2.5             Standard Performance EvaluationCorporation (SPEC)

2.5.1 Free and Open Source Benchmark Kits

As the benchmarkkits used by various database vendors are proprietary, several open sourcedatabase benchmark kits have been written to implement not only the industrystandard workloads, but others as well.

* Open Source Development Labs Database Test Suite

The OSDL DBT suite is the mostcomprehensive of all open source benchmark kits. While its benchmarkimplementations are based on TPC standards, they differ in someareas and are not certified; making a comparison between a certified TPC-Cresult and OSDL's implementation of TPC-C (DBT⑵) impossible. As the originalOSDL DBT suite lacks Oracle support, Oracle has added and released it as partof the Oracle Linux Test kit.

* PolePosition

PolePosition is an open sourcebenchmark test suite to compare database engines and object-relationalmapping technology.

* jTPCC

jTPCC is an open source Java implementation ofthe TPC-C benchmark. While it claims compliance with TPC-C, it is not.

* BenchmarkSQL

BenchmarkSQL is an open source,easy to use JDBC benchmark which closely resembles the TPC-C standard for OLTP;it was originally based on jTPCC.

* Bristlecone

Bristlecone is a simpleSELECT+INSERT/UPDATE/DELETE throughput and response-time benchmark written in Java.

* FinTime

FinTime is an open source set ofdata and queries which reflects the needs of financial analysts who arestudying patterns in stock market data, but it should appealto the designers of any system that has pretensions of handling ordered datawell.

* Java TPC-W Implementation

This open-source implementation is basedon the TPC-W specification version 1.0.1. Itstrays from the official benchmark specification in a fewminor areas, which are specified in documentationincluded with the distribution. This version itself lacks Oracle support, butit can be found on the web.

* Swingbench

Swingbench is a free loadgenerator (and benchmarks) designed to stress test an Oracle database(9i,10g,11g).

* OpenLink ODBC Bench

OpenLink ODBC Bench is an open-sourceODBC Benchmarking tool providing real-time comparative benchmarking for ODBCDrivers, Database Engines, and Operating Systems combinations. The Benchmarks in thisapplication are loosely based on the TPC-A and TPC-C standard benchmarks, withmodifications to specifically test the performance of an ODBC Driver and/orDatabase Engine in a client/server environment.

* OpenLink JDBC Bench

Similar to OpenLink's ODBC Bench, this is a JDBC version.

* TPCC-UVa

A free, open-source implementation of the TPC-C Benchmark.

* Hammerora

An open source load generation tool.

2.5.2 Commercial Benchmark Kits

There are several industry standard benchmarks.

* Official Oracle Benchmark Kits

Similar to Microsoft and IBM, the official benchmark kits used byOracle can only be obtained under certain conditions. For more information,contact your Oracle Consulting/Partner/Support representative.

* Real Application Testing

While many wouldn't think of Real Application Testing asa traditional benchmark kit, it does perform workload execution primarily forthe real world benchmarking case.

* Quest Software'sBenchmark Factory for Databases

Benchmark Factory for Databases is aworkload simulation and scalability testing tool that stress tests yourenvironment by simulating users and transactions on the database. You caneither replay production workload or use synthetic workload to test the limitsof your database.

* SPEC jAppServer Benchmark Kit

This can be acquired directly from SPEC.

2.6             Benchmarking-related Books

There are several good books regarding benchmarking. The best I'vefound include the following:

* The Benchmark Handbook: For Database andTransaction Processing Systems (Jim Gray)

A great, albeit outdated book. While several of the benchmarks itdiscusses have been deprecated, it presents good concepts and backgroundinformation regarding benchmarking.

* Performance Analysis of Transaction ProcessingSystems (Wilbur H. Highleyman)

While out of print, this bookpresents both plain-text and math-heavy concepts of queueing and performanceanalysis techniques oriented for OLTP.

* Database Benchmarking: Practical Methods forOracle & SQL Server

This book explores all aspects of database benchmarking and showsyou a real-world approach that ensures that you are prepared forwhatever the future brings to your mission-critical database.

--------------------------------------------------------------------------------------------

版权所有，文章制止转载，否则追究法律责任!

AboutDave:

--------------------------------------------------------------------------------------------

QQ:     251097186

Email:    tianlesoftware@gmail.com

Blog:    http://blog.csdn.net/tianlesoftware

Weibo:    http://weibo.com/tianlesoftware

Twitter:  http://twitter.com/tianlesoftware

Facebook: http://www.facebook.com/tianlesoftware

Linkedin: http://cn.linkedin.com/in/tianlesoftware

Dave 的QQ群：

--------------------------------------------------------------------------------------------

注意:加群必须注明表空间和数据文件关系 | 不要重复加群

CNDBA_1: 104207940 (满)    CNDBA_2: 62697716 (满)   CNDBA_3: 283816689

CNDBA_4: 391125754   CNDBA_5: 62697850    CNDBA_6: 62697977   CNDBA_7: 142216823（满）

生活不易，码农辛苦
如果您觉得本网站对您的学习有所帮助,可以手机扫描二维码进行捐赠