Python 熊猫：根据来自同一列的情侣分组

假设我有一个包含以下数据的表（数据框）：

| user     | food          |
|:--------:|:-------------:|
| 'A'      | 'meat'        | 
| 'A'      | 'carrot'      |
| 'A'      | 'candy'       |
| 'B'      |  'meat'       |
| 'B'      |  'carrot'      |
| 'C'      |  'meat'       |
| 'C'      |  'carrot'     |

代码：

我想为每两种食物建立一个表格，告诉我拥有这些食物的用户数量：

| food 1     | food 2        |  num users | 
|:----------:|:-------------:|:----------:| 
| 'meat'     | 'carrot'      | 3          | 
| 'meat'     | 'candy'       | 1          | 
| 'carrot'     | 'candy'       | 1          | 

有办法做到这一点吗？

您可以先使用：

然后按

列表理解进行计数

：

from  itertools import combinations

L = [(x[0], x[1],(df[list(x)] == 1).all(1).sum()) for x in list(combinations(df.columns, 2))]
print (L)
[('candy', 'carrot', 1), ('candy', 'meat', 1), ('carrot', 'meat', 3)]

df = pd.DataFrame(L, columns=['food 1','food 2','num users'])
print (df)
   food 1  food 2  num users
0   candy  carrot          1
1   candy    meat          1
2  carrot    meat          3

您可以尝试以下方法：

food_pairs = [("meat", "carrot"), ("meat", "candy")]

food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}

out = pd.DataFrame(
    ((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
    columns=["food1", "food2", "num users"]
)

1000次试验的平均运行时间为

0.00256s

可伸缩性测试代码：

import itertools
import math
import pandas as pd
from random import shuffle
from timeit import time

SIZE_OF_PAIRS = 2
NUM_FOODS = 50
NUM_USERS = 1000
NUM_RECORDS = 100000

foods = (list(range(NUM_FOODS)) * (math.ceil(NUM_RECORDS/NUM_FOODS)))[:NUM_RECORDS]
users = (list(range(NUM_USERS)) * (math.ceil(NUM_RECORDS/NUM_USERS)))[:NUM_RECORDS]

shuffle(foods)
shuffle(users)

df = pd.DataFrame({"user": users, "food": foods})

food_pairs = pd.Series([*itertools.combinations(df.food.unique(), SIZE_OF_PAIRS)])

start = time.time()

food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}
out = pd.DataFrame(
    ((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
    columns=[*["food" + str(i) for i in range(SIZE_OF_PAIRS)], "num users"]
)

print("Time taken: {}s".format(time.time() - start))

---

为什么肉要3个？对不起，我数错了！在我给出的样本数据框上测量时间了吗？@martin Yeah，不包括

食物对

创建。它能合理地扩展到更大的数据帧吗？@martin我还没有对复杂性做过任何透彻的分析，或者是否有更有效的方法，但我已经编辑了答案，包括一个测试代码，您可以修改参数，看看性能是否可以接受。@martin顺便说一下，如果您的数据类型是int而不是string，那么它的运行速度大约要快20倍。还没有调查为什么这是一个值得考虑的问题。谢谢，它在大型数据文件上很慢，但是很好的解决方案。

food_pairs = [("meat", "carrot"), ("meat", "candy")]

food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}

out = pd.DataFrame(
    ((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
    columns=["food1", "food2", "num users"]
)

import itertools
import math
import pandas as pd
from random import shuffle
from timeit import time

SIZE_OF_PAIRS = 2
NUM_FOODS = 50
NUM_USERS = 1000
NUM_RECORDS = 100000

foods = (list(range(NUM_FOODS)) * (math.ceil(NUM_RECORDS/NUM_FOODS)))[:NUM_RECORDS]
users = (list(range(NUM_USERS)) * (math.ceil(NUM_RECORDS/NUM_USERS)))[:NUM_RECORDS]

shuffle(foods)
shuffle(users)

df = pd.DataFrame({"user": users, "food": foods})

food_pairs = pd.Series([*itertools.combinations(df.food.unique(), SIZE_OF_PAIRS)])

start = time.time()

food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}
out = pd.DataFrame(
    ((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
    columns=[*["food" + str(i) for i in range(SIZE_OF_PAIRS)], "num users"]
)

print("Time taken: {}s".format(time.time() - start))