Benchmarks
Date Lang Batch Benchmark Mean Z-Score Error
2023-01-24 22:19 Python dataframe-to-table type_integers 0.010 s 0.586
2023-01-24 22:23 Python dataset-read async=True, pre_buffer=true, nyctaxi_multi_parquet_s3 80.486 s 0.085
2023-01-24 22:15 Python csv-read uncompressed, arrow_table, streaming, fanniemae_2016Q4 13.539 s 1.157
2023-01-24 22:13 Python csv-read gzip, arrow_table, file, fanniemae_2016Q4 5.772 s 0.921
2023-01-24 22:16 Python csv-read gzip, arrow_table, streaming, nyctaxi_2010-01 11.375 s -1.528
2023-01-24 22:19 Python dataframe-to-table type_strings 0.428 s -0.007
2023-01-24 22:16 Python csv-read uncompressed, arrow_table, file, nyctaxi_2010-01 1.131 s 0.097
2023-01-24 22:19 Python dataframe-to-table type_nested 2.960 s 0.142
2023-01-24 22:19 Python dataset-filter nyctaxi_2010-01 1.035 s -0.094
2023-01-24 22:14 Python csv-read gzip, arrow_table, streaming, fanniemae_2016Q4 13.523 s 1.344
2023-01-24 22:14 Python csv-read uncompressed, arrow_table, file, fanniemae_2016Q4 1.238 s 0.238
2023-01-24 22:16 Python csv-read gzip, arrow_table, file, nyctaxi_2010-01 8.433 s 0.596
2023-01-24 22:17 Python csv-read uncompressed, arrow_table, streaming, nyctaxi_2010-01 11.374 s -1.681
2023-01-24 22:19 Python dataframe-to-table type_dict 0.011 s 2.251
2023-01-24 22:28 Python dataset-read async=True, pre_buffer=false, nyctaxi_multi_parquet_s3 83.925 s 0.176
2023-01-24 22:19 Python dataframe-to-table chi_traffic_2020_Q1 20.932 s 0.539
2023-01-24 22:19 Python dataframe-to-table type_floats 0.010 s 0.153
2023-01-24 22:39 Python dataset-read async=True, nyctaxi_multi_ipc_s3 222.442 s 0.149
2023-01-24 22:39 Python dataset-selectivity 1%, nyctaxi_multi_parquet_s3 1.259 s -0.302
2023-01-24 22:39 Python dataset-select nyctaxi_multi_parquet_s3_repartitioned 1.626 s -1.731
2023-01-24 22:39 Python dataset-selectivity 10%, nyctaxi_multi_parquet_s3 1.251 s -0.013
2023-01-24 22:39 Python dataset-selectivity 100%, nyctaxi_multi_parquet_s3 1.350 s -0.223
2023-01-24 22:39 Python dataset-selectivity 1%, nyctaxi_multi_ipc_s3 2.044 s 0.191
2023-01-24 22:40 Python dataset-selectivity 1%, chi_traffic_2020_Q1 1.185 s -0.548
2023-01-24 22:39 Python dataset-selectivity 10%, nyctaxi_multi_ipc_s3 2.044 s 0.178
2023-01-24 22:40 Python dataset-selectivity 100%, nyctaxi_multi_ipc_s3 1.672 s 0.212
2023-01-24 22:40 Python dataset-selectivity 10%, chi_traffic_2020_Q1 1.214 s -0.061
2023-01-24 22:40 Python dataset-serialize parquet, 1pc, nyctaxi_multi_parquet_s3 0.304 s 1.229
2023-01-24 22:40 Python dataset-serialize feather, 1pc, nyctaxi_multi_parquet_s3 0.023 s -1.784
2023-01-24 22:40 Python dataset-selectivity 100%, chi_traffic_2020_Q1 1.142 s 0.164
2023-01-24 22:40 Python dataset-serialize arrow, 1pc, nyctaxi_multi_parquet_s3 0.023 s -0.009
2023-01-24 22:40 Python dataset-serialize csv, 1pc, nyctaxi_multi_parquet_s3 0.730 s 1.301
2023-01-24 22:40 Python dataset-serialize parquet, 10pc, nyctaxi_multi_parquet_s3 2.905 s 1.486
2023-01-24 22:40 Python dataset-serialize feather, 10pc, nyctaxi_multi_parquet_s3 0.200 s -1.608
2023-01-24 22:40 Python dataset-serialize arrow, 10pc, nyctaxi_multi_parquet_s3 0.199 s 0.386
2023-01-24 22:41 Python dataset-serialize csv, 10pc, nyctaxi_multi_parquet_s3 7.257 s 1.300
2023-01-24 22:44 Python dataset-serialize parquet, 100pc, nyctaxi_multi_parquet_s3 30.017 s 1.572
2023-01-24 22:45 Python dataset-serialize arrow, 100pc, nyctaxi_multi_parquet_s3 2.151 s -0.698
2023-01-24 22:45 Python dataset-serialize feather, 100pc, nyctaxi_multi_parquet_s3 2.157 s -2.215
2023-01-24 22:52 Python dataset-serialize parquet, 1pc, nyctaxi_multi_ipc_s3 0.282 s 1.320
2023-01-24 22:53 Python dataset-serialize parquet, 10pc, nyctaxi_multi_ipc_s3 3.000 s 1.154
2023-01-24 22:52 Python dataset-serialize csv, 100pc, nyctaxi_multi_parquet_s3 73.303 s 1.293
2023-01-24 22:53 Python dataset-serialize csv, 1pc, nyctaxi_multi_ipc_s3 0.835 s 1.259
2023-01-24 22:52 Python dataset-serialize arrow, 1pc, nyctaxi_multi_ipc_s3 0.026 s -1.748
2023-01-24 22:52 Python dataset-serialize feather, 1pc, nyctaxi_multi_ipc_s3 0.026 s 0.171
2023-01-24 22:53 Python dataset-serialize feather, 10pc, nyctaxi_multi_ipc_s3 0.225 s 0.477
2023-01-24 22:53 Python dataset-serialize arrow, 10pc, nyctaxi_multi_ipc_s3 0.226 s -0.246
2023-01-24 22:54 Python dataset-serialize csv, 10pc, nyctaxi_multi_ipc_s3 8.391 s 1.227
2023-01-24 23:16 Python file-write lz4, feather, dataframe, nyctaxi_2010-01 3.232 s -0.403
2023-01-24 22:57 Python dataset-serialize parquet, 100pc, nyctaxi_multi_ipc_s3 30.427 s 1.162
2023-01-24 22:58 Python dataset-serialize feather, 100pc, nyctaxi_multi_ipc_s3 2.410 s 0.004
2023-01-24 22:57 Python dataset-serialize arrow, 100pc, nyctaxi_multi_ipc_s3 2.406 s 0.464
2023-01-24 23:08 Python file-read snappy, parquet, table, nyctaxi_2010-01 0.950 s -0.228
2023-01-24 23:14 Python file-write uncompressed, parquet, dataframe, nyctaxi_2010-01 7.355 s 0.084
2023-01-24 23:14 Python file-write snappy, parquet, table, nyctaxi_2010-01 7.737 s 0.217
2023-01-24 23:27 R dataframe-to-table type_dict, R 0.057 s -0.315
2023-01-24 23:28 R file-read uncompressed, parquet, dataframe, fanniemae_2016Q4, R 1.590 s -0.483
2023-01-24 23:30 R file-read lz4, feather, table, nyctaxi_2010-01, R 0.583 s 0.145
2023-01-24 23:38 R file-write uncompressed, feather, dataframe, fanniemae_2016Q4, R 9.038 s -0.409
2023-01-24 23:43 R file-write snappy, parquet, table, nyctaxi_2010-01, R 6.667 s 0.722
2023-01-24 23:47 R file-write lz4, feather, table, nyctaxi_2010-01, R 1.473 s -0.672
2023-01-24 23:08 Python file-read lz4, feather, table, nyctaxi_2010-01 0.669 s 0.298
2023-01-24 23:11 Python file-write snappy, parquet, dataframe, fanniemae_2016Q4 20.228 s 0.890
2023-01-24 23:15 Python file-write uncompressed, feather, table, nyctaxi_2010-01 2.762 s -0.298
2023-01-24 23:26 R dataframe-to-table type_strings, R 0.536 s -0.359
2023-01-24 23:27 R dataframe-to-table type_integers, R 0.010 s -1.561
2023-01-24 23:27 R dataframe-to-table type_nested, R 0.576 s -0.470
2023-01-24 23:34 R file-write snappy, parquet, table, fanniemae_2016Q4, R 10.127 s 0.765
2023-01-24 23:36 R file-write uncompressed, feather, table, fanniemae_2016Q4, R 2.962 s -0.123
2023-01-24 23:46 R file-write uncompressed, feather, dataframe, nyctaxi_2010-01, R 2.176 s 0.209
2023-01-24 23:48 R file-write lz4, feather, dataframe, nyctaxi_2010-01, R 2.116 s -2.793
2023-01-24 23:07 Python file-read snappy, parquet, dataframe, fanniemae_2016Q4 1.518 s -0.579
2023-01-24 23:07 Python file-read uncompressed, feather, dataframe, fanniemae_2016Q4 5.729 s -0.116
2023-01-24 23:08 Python file-read snappy, parquet, dataframe, nyctaxi_2010-01 0.939 s 0.067
2023-01-24 23:08 Python file-read uncompressed, feather, dataframe, nyctaxi_2010-01 1.578 s 0.277
2023-01-24 23:26 R dataframe-to-table chi_traffic_2020_Q1, R 4.355 s 0.225
2023-01-24 23:28 R file-read uncompressed, feather, table, fanniemae_2016Q4, R 0.314 s 0.201
2023-01-24 23:38 R file-write lz4, feather, table, fanniemae_2016Q4, R 1.497 s -2.045
2023-01-24 23:44 R file-write snappy, parquet, dataframe, nyctaxi_2010-01, R 7.719 s 0.746
2023-01-24 23:48 R partitioned-dataset-filter small_no_files, dataset-taxi-parquet, R 0.252 s 0.188
2023-01-24 23:06 Python file-read uncompressed, parquet, table, fanniemae_2016Q4 1.651 s -0.021
2023-01-24 23:07 Python file-read lz4, feather, table, fanniemae_2016Q4 0.822 s -0.313
2023-01-24 23:08 Python file-read uncompressed, feather, table, nyctaxi_2010-01 0.930 s 0.235
2023-01-24 23:10 Python file-write uncompressed, parquet, dataframe, fanniemae_2016Q4 20.142 s 0.348
2023-01-24 23:12 Python file-write uncompressed, feather, table, fanniemae_2016Q4 6.326 s -0.254
2023-01-24 23:13 Python file-write lz4, feather, dataframe, fanniemae_2016Q4 10.955 s -0.592
2023-01-24 23:08 Python file-read uncompressed, parquet, dataframe, nyctaxi_2010-01 0.987 s -0.059
2023-01-24 23:15 Python file-write snappy, parquet, dataframe, nyctaxi_2010-01 9.235 s 0.158
2023-01-24 23:29 R file-read snappy, parquet, dataframe, nyctaxi_2010-01, R 0.922 s -0.323
2023-01-24 23:30 R file-read lz4, feather, dataframe, nyctaxi_2010-01, R 0.898 s 0.207
2023-01-24 23:40 R file-write lz4, feather, dataframe, fanniemae_2016Q4, R 7.500 s -0.845
2023-01-24 23:07 Python file-read snappy, parquet, table, fanniemae_2016Q4 1.526 s -0.770
2023-01-24 23:08 Python file-read uncompressed, parquet, table, nyctaxi_2010-01 0.985 s 0.089
2023-01-24 23:09 Python file-write uncompressed, parquet, table, fanniemae_2016Q4 10.560 s 0.507
2023-01-24 23:13 Python file-write uncompressed, feather, dataframe, fanniemae_2016Q4 15.341 s -0.355
2023-01-24 23:30 R file-read uncompressed, feather, table, nyctaxi_2010-01, R 0.216 s 0.161
2023-01-24 23:07 Python file-read lz4, feather, dataframe, fanniemae_2016Q4 4.310 s -1.707
2023-01-24 23:10 Python file-write snappy, parquet, table, fanniemae_2016Q4 10.750 s 0.756
2023-01-24 23:16 Python wide-dataframe use_legacy_dataset=false 0.513 s 0.253
2023-01-24 23:28 R file-read uncompressed, feather, dataframe, fanniemae_2016Q4, R 0.570 s 0.141
2023-01-24 23:29 R file-read lz4, feather, dataframe, fanniemae_2016Q4, R 0.861 s 0.155
2023-01-24 23:48 R partitioned-dataset-filter vignette, dataset-taxi-parquet, R 0.625 s -3.751
2023-01-24 23:07 Python file-read uncompressed, feather, table, fanniemae_2016Q4 2.325 s 0.179
2023-01-24 23:08 Python file-read lz4, feather, dataframe, nyctaxi_2010-01 1.318 s 0.437
2023-01-24 23:15 Python file-write uncompressed, feather, dataframe, nyctaxi_2010-01 4.168 s -0.351
2023-01-24 23:06 Python dataset-serialize csv, 100pc, nyctaxi_multi_ipc_s3 83.229 s 1.253
2023-01-24 23:06 Python file-read uncompressed, parquet, dataframe, fanniemae_2016Q4 1.761 s -1.887
2023-01-24 23:13 Python file-write lz4, feather, table, fanniemae_2016Q4 1.867 s -0.258
2023-01-24 23:14 Python file-write uncompressed, parquet, table, nyctaxi_2010-01 5.835 s 0.304
2023-01-24 23:16 Python file-write lz4, feather, table, nyctaxi_2010-01 1.821 s -0.511
2023-01-24 23:27 R file-read uncompressed, parquet, table, fanniemae_2016Q4, R 1.337 s -0.606
2023-01-24 23:28 R file-read snappy, parquet, table, fanniemae_2016Q4, R 1.344 s -0.908
2023-01-24 23:29 R file-read uncompressed, parquet, dataframe, nyctaxi_2010-01, R 0.919 s -0.243
2023-01-24 23:30 R file-read uncompressed, feather, dataframe, nyctaxi_2010-01, R 0.813 s 0.177
2023-01-24 23:41 R file-write uncompressed, parquet, table, nyctaxi_2010-01, R 4.923 s 0.967
2023-01-24 23:45 R file-write uncompressed, feather, table, nyctaxi_2010-01, R 1.311 s -0.047
2023-01-24 23:16 Python wide-dataframe use_legacy_dataset=true 0.378 s -0.003
2023-01-24 23:28 R file-read snappy, parquet, dataframe, fanniemae_2016Q4, R 1.595 s -0.705
2023-01-24 23:28 R file-read lz4, feather, table, fanniemae_2016Q4, R 0.593 s 0.181
2023-01-24 23:29 R file-read snappy, parquet, table, nyctaxi_2010-01, R 0.575 s -0.250
2023-01-24 23:33 R file-write uncompressed, parquet, dataframe, fanniemae_2016Q4, R 16.684 s 0.659
2023-01-24 23:36 R file-write snappy, parquet, dataframe, fanniemae_2016Q4, R 17.125 s 0.730
2023-01-24 23:27 R dataframe-to-table type_floats, R 0.013 s 0.663
2023-01-24 23:29 R file-read uncompressed, parquet, table, nyctaxi_2010-01, R 0.573 s -0.211
2023-01-24 23:31 R file-write uncompressed, parquet, table, fanniemae_2016Q4, R 9.718 s 0.706
2023-01-24 23:42 R file-write uncompressed, parquet, dataframe, nyctaxi_2010-01, R 5.769 s 0.661
2023-01-24 23:48 R partitioned-dataset-filter payment_type_3, dataset-taxi-parquet, R 1.613 s -0.850
2023-01-24 23:49 R partitioned-dataset-filter dims, dataset-taxi-parquet, R 0.587 s 0.951
2023-01-24 23:59 JavaScript Iterate Vector int32Array 0.002 s 0.622
2023-01-24 23:59 JavaScript vectorFromArray dictionary 0.016 s 1.090
2023-01-24 23:59 JavaScript Iterate Vector uint16Array 0.002 s 0.848
2023-01-24 23:59 JavaScript Iterate Vector int16Array 0.002 s 0.528
2023-01-24 23:59 JavaScript vectorFromArray booleans 0.017 s 0.663
2023-01-24 23:59 JavaScript Iterate Vector dictionary 0.004 s 1.193
2023-01-24 23:59 JavaScript Spread Vector uint8Array 0.006 s 0.593
2023-01-24 23:59 JavaScript Spread Vector uint32Array 0.007 s 0.824
2023-01-24 23:59 JavaScript Table 1,000,000, tracks 0.271 s 0.173
2023-01-24 23:59 JavaScript toArray Vector uint16Array
2023-01-24 23:59 JavaScript toArray Vector int16Array
2023-01-24 23:59 JavaScript toArray Vector int64Array
2023-01-24 23:59 JavaScript get Vector string 0.125 s -0.485
2023-01-24 23:59 JavaScript Iterate Vector int64Array 0.004 s 1.096
2023-01-24 23:59 JavaScript toArray Vector string 0.144 s 0.540
2023-01-24 23:59 JavaScript get Vector uint16Array 0.003 s -0.294
2023-01-24 23:59 JavaScript Slice vectors lng, 1,000,000, Float32, tracks 0.000 s
2023-01-24 23:59 JavaScript vectorFromArray numbers 0.016 s 0.172
2023-01-24 23:59 JavaScript Iterate Vector uint64Array 0.004 s 0.594
2023-01-24 23:59 JavaScript Iterate Vector string 0.125 s 0.532
2023-01-24 23:59 JavaScript Spread Vector int16Array 0.007 s 0.088
2023-01-24 23:59 JavaScript Spread Vector int64Array 0.012 s 0.058
2023-01-24 23:59 JavaScript Iterate Vector uint8Array 0.002 s -0.115
2023-01-24 23:59 JavaScript Iterate Vector uint32Array 0.002 s 0.434
2023-01-24 23:59 JavaScript Iterate Vector float32Array 0.002 s 0.851
2023-01-24 23:59 JavaScript Iterate Vector numbers 0.002 s 0.503
2023-01-24 23:59 JavaScript Iterate Vector int8Array 0.002 s 0.774
2023-01-24 23:59 JavaScript toArray Vector int32Array
2023-01-24 23:59 JavaScript toArray Vector float32Array
2023-01-24 23:59 JavaScript get Vector uint32Array 0.003 s -0.203
2023-01-24 23:59 JavaScript Table tracks, 1,000,000 0.095 s 0.603
2023-01-24 23:59 JavaScript Iterate Vector float64Array 0.002 s 0.553
2023-01-24 23:59 JavaScript Iterate Vector booleans 0.004 s 0.136
2023-01-24 23:59 JavaScript toArray Vector uint64Array
2023-01-24 23:59 JavaScript toArray Vector float64Array
2023-01-24 23:59 JavaScript Iterate vectors origin, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.040 s 0.105
2023-01-24 23:59 JavaScript Spread Vector uint16Array 0.007 s -0.080
2023-01-24 23:59 JavaScript Spread Vector uint64Array 0.012 s 0.456
2023-01-24 23:59 JavaScript Spread Vector string 0.145 s 0.290
2023-01-24 23:59 JavaScript get Vector int16Array 0.003 s -0.009
2023-01-24 23:59 JavaScript get Vector float64Array 0.002 s -0.479
2023-01-24 23:59 JavaScript Spread Vector int8Array 0.006 s 0.223
2023-01-24 23:59 JavaScript Spread Vector int32Array 0.006 s 0.872
2023-01-24 23:59 JavaScript toArray Vector numbers
2023-01-24 23:59 JavaScript get Vector uint8Array 0.003 s -1.157
2023-01-24 23:59 JavaScript get Vector int32Array 0.003 s 0.014
2023-01-24 23:59 JavaScript Slice vectors destination, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.000 s
2023-01-24 23:59 JavaScript Spread Vector float32Array 0.008 s 0.801
2023-01-24 23:59 JavaScript Spread Vector numbers 0.008 s 0.458
2023-01-24 23:59 JavaScript get Vector numbers 0.002 s -0.708
2023-01-24 23:59 JavaScript Get values by index lat, 1,000,000, Float32, tracks 0.030 s -0.537
2023-01-24 23:59 JavaScript Iterate vectors lat, 1,000,000, Float32, tracks 0.023 s 0.924
2023-01-24 23:59 JavaScript Spread Vector float64Array 0.008 s 0.750
2023-01-24 23:59 JavaScript Spread Vector booleans 0.010 s 0.197
2023-01-24 23:59 JavaScript get Vector int64Array 0.003 s -0.357
2023-01-24 23:59 JavaScript Iterate vectors destination, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.039 s 0.251
2023-01-24 23:59 JavaScript Slice toArray vectors destination, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.108 s -0.520
2023-01-24 23:59 JavaScript Table tracks, 1,000,000 0.050 s -0.125
2023-01-24 23:59 JavaScript Spread Vector dictionary 0.010 s -0.029
2023-01-24 23:59 JavaScript toArray Vector uint32Array
2023-01-24 23:59 JavaScript toArray Vector int8Array
2023-01-24 23:59 JavaScript toArray Vector dictionary 0.010 s -0.026
2023-01-24 23:59 JavaScript toArray Vector uint8Array
2023-01-24 23:59 JavaScript Get values by index origin, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.039 s 0.247
2023-01-24 23:59 JavaScript Slice toArray vectors origin, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.108 s 0.170
2023-01-24 23:59 JavaScript Spread vectors lat, 1,000,000, Float32, tracks 0.188 s 0.021
2023-01-24 23:59 JavaScript toArray Vector booleans 0.010 s 0.744
2023-01-24 23:59 JavaScript get Vector uint64Array 0.003 s -0.417
2023-01-24 23:59 JavaScript Get values by index lng, 1,000,000, Float32, tracks 0.030 s -0.214
2023-01-24 23:59 JavaScript Get values by index destination, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.039 s 0.217
2023-01-24 23:59 JavaScript Iterate vectors lng, 1,000,000, Float32, tracks 0.023 s 0.941
2023-01-24 23:59 JavaScript Spread vectors lng, 1,000,000, Float32, tracks 0.188 s -0.008
2023-01-24 23:59 JavaScript get Vector int8Array 0.003 s -0.066
2023-01-24 23:59 JavaScript Slice vectors lat, 1,000,000, Float32, tracks 0.000 s
2023-01-24 23:59 JavaScript Spread vectors origin, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.108 s -0.357
2023-01-24 23:59 JavaScript Table Direct Count lat, 1,000,000, gt, Float32, 0, tracks 0.032 s 0.473
2023-01-24 23:59 JavaScript Table Direct Count origin, 1,000,000, eq, Dictionary<Int8, Utf8>, Seattle, tracks 0.032 s 0.420
2023-01-24 23:59 JavaScript get Vector float32Array 0.002 s 0.568
2023-01-24 23:59 JavaScript get Vector dictionary 0.002 s 0.718
2023-01-24 23:59 JavaScript Parse read recordBatches, tracks 0.000 s -1.559
2023-01-24 23:59 JavaScript Slice toArray vectors lat, 1,000,000, Float32, tracks 0.000 s 0.131
2023-01-24 23:59 JavaScript Slice vectors origin, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.000 s
2023-01-24 23:59 JavaScript get Vector booleans 0.002 s -0.141
2023-01-24 23:59 JavaScript Spread vectors destination, 1,000,000, Dictionary<Int8, Utf8>, tracks 0.108 s 0.384
2023-01-24 23:59 JavaScript Table tracks, 1,000,000 0.277 s -0.477
2023-01-24 23:59 JavaScript Parse write recordBatches, tracks 0.002 s -1.399
2023-01-24 23:59 JavaScript Slice toArray vectors lng, 1,000,000, Float32, tracks 0.000 s -0.585
2023-01-24 23:59 JavaScript Table Direct Count lng, 1,000,000, gt, Float32, 0, tracks 0.032 s 0.455