Filter number of errors in build server query prior to the query

I am using TFS API to query a rather large build server to obtain builds that match the provided IBuildDetailSpec and am encountering a problem when pulling the list of errors. If I query for a build that has a lot of errors (10k+), the execution time of the query slows down dramatically. Is there a way to obtain the number of errors prior to the build server query so that these builds can be filtered out?

My code:

var buildDetailSpec = buildOptions.buildServer.CreateBuildDetailSpec(teamProject.Text, buildDefinition.Text);
//Narrow down search results in order to increase query performance
buildDetailSpec.InformationTypes = new[] {"BuildError"};
buildDetailSpec.MinFinishTime = DateTime.Now - new TimeSpan(2, 0, 0, 0);
buildDetailSpec.MaxBuildsPerDefinition = 50;
buildDetailSpec.QueryOrder = BuildQueryOrder.StartTimeDescending;
var builds = buildOptions.buildServer.QueryBuilds(buildDetailSpec).Builds;