C# 嵌套foreach循环的性能问题
我不知道我面临的问题是由于嵌套循环还是其他原因(错误代码、大数据)。让我解释一下,希望有人能帮我弄明白:- 我正在开发一个WindowsPhone8应用程序。启动时,应用程序对3个不同的页面进行三次httpclient调用,然后将每个响应(html)转换为xml,然后再转换为json数据(OBS!!!这很好,没有问题)。在这个阶段,我想从这3个json中提取数据,并将提取的数据合并成一个新的json,稍后保存到IsolatedStorage中。为了进行提取,我使用了多个foreach循环,甚至有时使用linq(我认为这就是问题所在)。每个foreach循环可能进行50到>500次迭代。提取过程几乎需要2分钟,我认为这是非常重要的。下面您可以看到我的代码中的代码片段:-C# 嵌套foreach循环的性能问题,c#,linq,foreach,json.net,C#,Linq,Foreach,Json.net,我不知道我面临的问题是由于嵌套循环还是其他原因(错误代码、大数据)。让我解释一下,希望有人能帮我弄明白:- 我正在开发一个WindowsPhone8应用程序。启动时,应用程序对3个不同的页面进行三次httpclient调用,然后将每个响应(html)转换为xml,然后再转换为json数据(OBS!!!这很好,没有问题)。在这个阶段,我想从这3个json中提取数据,并将提取的数据合并成一个新的json,稍后保存到IsolatedStorage中。为了进行提取,我使用了多个foreach循环,甚至有
public async Task LoadData()
{
//baseUri definition....
await CookieHandler.GetCookies(baseUri);
_reqPList=new RequestResponse();
await _reqPList.GetResponse(baseUri, pList); // First page request
XmlConvertor.ConvertToXml(_reqPList.Response); //First page convert to xml
var phoneListResponse = XmlConvertor.XmlString;
//Second and third page request and convert to xml
var handler = new DataHandler();
await handler.phoneListHandler(phoneListResponse); //First page convert to Json
//Second and third page convert to Json
var jObject = JObject.Parse(handler.PhoneListJson);
var jCollection = jObject["tbody"]["tr"].Children();
//The start of extracting data
using (var stream = new MemoryStream())
using (var streamWriter=new StreamWriter(stream))
using (_jsonW = new JsonTextWriter(streamWriter))
{
var watch = Stopwatch.StartNew();
_jsonW.WriteStartArray();
foreach (var item in jCollection) //First foreach loop (100 iteration) using the first page
{
var arr = item.Value<JArray>("td");
if (arr == null) continue;
_jsonW.Formatting = Formatting.Indented;
_jsonW.WriteStartObject();
_jsonW.WritePropertyName("Name");
_jsonW.WriteValue(arr[0].Value<string>());
#region Info Array
_jsonW.WritePropertyName("Info");
_jsonW.WriteStartArray();
// Do some extraction
_jsonW.WriteEndArray();
#endregion
#region Schedule Array
await ScheduleArrayAsync(item, handler.ScheduleJson, handler); // This method extract data from 2nd page
#endregion
#region Current activity
CurrActivityAsync(item, handler.CurrActivitiesJson); // This method extract data from 3rd page
#endregion
_jsonW.WriteEndObject();
}
_jsonW.WriteEndArray();
watch.Stop();
var j = watch.ElapsedMilliseconds;
await streamWriter.FlushAsync();
stream.Position = 0;
using (var streamReader=new StreamReader(stream))
{
var json = await streamReader.ReadToEndAsync();
}
}
this.IsDataLoaded = true;
}
private void CurrActivityAsync(JToken token, string jString) This method is called 1 time only
{
var arr = token.Value<JArray>("td");
var personNameArray = arr[0].Value<string>().Split(',');
if (!_owner.Contains(personNameArray[0])) return;
var jObject = JObject.Parse(jString);
var jCollection = jObject["tbody"]["tr"].Children();
_jsonW.WritePropertyName("Current activity");
_jsonW.WriteStartArray();
foreach (var array in from item in jCollection // Second foreach loop (Almost 60 iteration)
let array = item.Value<JObject>().Value<JArray>("td")
where array != null && item.Children().Count() > 1
select array)
{
_jsonW.WriteStartObject();
// do some extraction
_jsonW.WriteEndObject();
}
_jsonW.WriteEndArray();
}
private async Task ScheduleArrayAsync(JToken token, string jString, DataHandler handler) // This method is called 100 times.
{
var arr = token.Value<JArray>("td");
var personNameArray = arr[0].Value<string>().Split(',');
var jObject = JObject.Parse(jString);
var trList = jObject["tbody"]["tr"];
var selectedArray = default(JArray);
var weekNumber = trList.SelectMany(tr => tr.SelectTokens("td[1].table.tbody.tr", false))
.SelectMany(lk => lk.SelectTokens("td", false))
.Where(j => (!j.HasValues) && (j.Value<string>().Contains("v")))
.Select(lo => lo.Value<string>())
.ToArray();
var psTypeOne = trList.SelectMany(tr => tr.SelectTokens("td[0].a.#text", false))
.Where(hashText => (hashText.Value<string>().Contains(personNameArray[0])) &&
(hashText.Value<string>().Contains(personNameArray[1])))
.Select(td => td.Parent.Parent.Parent.Parent.Parent)
.ToArray();
if (psTypeOne != null)
{
_jsonW.WritePropertyName("Schedule");
_jsonW.WriteStartArray();
_jsonW.WriteStartObject();
_jsonW.WritePropertyName("Week Number");
_jsonW.WriteValue(weekNumber[0].Remove(0, 2));
_jsonW.WritePropertyName("Week days");
_jsonW.WriteStartArray();
foreach (var day in selectedArray) // 7 itereation
{
if (day.SelectToken("table", false) == null) continue;
_jsonW.WriteStartObject();
_jsonW.WritePropertyName("Day name");
var dayname = Convert.ToDateTime(day.Value<string>("@id").Remove(0, 8));
_jsonW.WriteValue(dayname.DayOfWeek.ToString());
_jsonW.WritePropertyName("Date");
_jsonW.WriteValue(day.Value<string>("@id").Remove(0, 8));
var assignments = day.SelectToken("table.tbody.tr", false);
_jsonW.WritePropertyName("Assignments");
_jsonW.WriteStartArray();
//Foreach loop 2 start or Foreach
foreach (var assignment in assignments) // between 1 and 5 iterations
{
//Some extractions
if (info[0].Contains("slotInfo")) //Here we do some new webrequests and extractions
{
await _reqPList.GetSlotInfoPost(info[1]);
XmlConvertor.ConvertToXml(_reqPList.Response);
addedInfo = XmlConvertor.XmlString;
await handler.InfoHandler(addedInfo);
var jO = JObject.Parse(handler.InfoJson);
addedJToken = jO.SelectToken("td");
// Some extractions
}
}
else
{
// Some code
}
}
//Foreach loop 2 end
//Some code to Close the Array
}
//Foreach loop 1 end
//some code to Close the Array
}
}
OBS!!!我知道json数据需要关闭,这只是为了说明一点。我认为从httprequest响应简单地遍历html可能更简单 使用serializable属性准确表示最终对象的自定义类,以及填充该类时调用javascript序列化程序将所有内容序列化为json对象(您可以使用联机将自定义类中的json转换为json,这样您就可以简单地将json反序列化为自定义对象,而不会出现问题,反之亦然) 还有另一件重要的事情:数据维度。如果您使用的数据较少,是否会更快?如果您使用的是大数据,则需要更多的时间做出响应,并需要更多的时间进行处理
您能告诉我们哪些是包含的页面,哪些是最终的json对象吗?handler.InfoHandler(addedInfo)做什么?是否有异步的原因?@siride handler.InfoHandler(addedInfo)是对异步方法的调用,用于将数据从xml转换为json。我分析的原始3页中没有这段数据。为什么在这里需要使用异步?通过这种方式比同步方式节省时间真的是性能密集型的吗?我以前试过htmlagility Pack,但它不适合我的需要。无论如何,我认为这不是问题所在。请看一下编辑后的信息。您将看到,将原始Json转换为我的Json并非易事,需要手动提取数据。如果你认为我误解了你的答案,请给我举个例子,让我更好地理解。
<tr class="odd">
<td class="user" onmouseover="userInfo('469');" onmouseout="userInfo(0);" onmousemove="moveSlotInfo();"> User One</td> // User One is needed
<td id="day-469-2014-03-31" style="vertical-align: top;">
<table class="ss" cellspacing="1">
<tbody>
<tr>
<td class="as"> </td>
</tr>
</tbody>
</table>
</td>
<td id="day-469-2014-04-01" style="vertical-align: top;"> // The date is needed
<table class="ss" cellspacing="1">
<tbody>
<tr>
<td class="as" style="color: #ffffff; background-color: #4040ff;" onmouseover="this.className=(document.week_vs_doctor.activityId.value==-1?'sd':'sp');slotInfo('177738',false);" onmouseout="this.className='as';slotInfo(0,false);" onmousemove="moveSlotInfo();">KAVAul</td> // KAVAul is needed
</tr>
</tbody>
</table>
</td>
[
{
"Name": "User Name",
"Info": [
{
"Signature": "un"
},
{
"Group": "Group1"
},
{
"E-mail": "user.name@group1.com"
}
],
"Schedule": [
{
"Week Number": "14",
"Week days": [
{
"Day name": "Monday",
"Date": "2014-03-31",
"Assignments": [
{
"Assignment name": "AAA",
"Assignment time": "07:30 - 16:00",
"Assignment provider": "BBB, 2014-03-07 16:42"
}