Python 为分层数据帧列创建数字标识符

我有一个Pandas数据框架，包含10多列数据和数百万行

三列构成一个具有三个不同级别的层次结构：

high

、

medium

和

low

。这三列包含不缺少数据的字符串。在整个组合层次结构中，每一列都按字典顺序排列，因此，例如

[“A…”、“B…”、“C…”]

位于

[“H…”、“A…”、“B…”之前

我想添加三个新的整数列：
高id
，
中id
，
低id
。这三个
X_id
列中的每一列都应该为每个数据帧行指定一个值。第一行的
X_id
列最初设置为1<当列的相应
X
值与前一行不同时，code>X\u id

列将递增，除非更高级别的值发生变化，从而将

X\u id

重置为1

纯Python实现示例：

rows = [
    ["high1", "med1", "low1"],
    ["high1", "med1", "low1"],
    ["high1", "med1", "low2"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low4"],
    ["high1", "med2", "low5"],
    ["high1", "med2", "low6"],
    ["high1", "med3", "low7"],
    ["high1", "med3", "low7"],
    ["high1", "med3", "low7"],
    ["high1", "med4", "low8"],
    ["high2", "med5", "low9"],
    ["high2", "med5", "lowA"],
    ["high2", "med5", "lowA"],
    ["high2", "med6", "lowB"],
    ["high3", "med4", "lowC"],
    ["high3", "med7", "low1"],
    ["high3", "med7", "lowD"],
    ["high3", "med7", "lowE"]]

high_id, medium_id, low_id = 1, 1, 1
ids = [[high_id, medium_id, low_id]]
previous_row = rows[0]

for row in rows[1:]:
    # Compare "high"
    if previous_row[0] != row[0]:
        high_id += 1
        medium_id = 1
        low_id = 1
    # Compare "medium"
    elif previous_row[1] != row[1]:
        medium_id += 1
        low_id = 1
    # Compare "low"
    elif previous_row[2] != row[2]:
        low_id += 1
    ids.append([high_id, medium_id, low_id])
    previous_row = row

for i, v in enumerate(rows):
    print(v + ids[i])

输出：

# high, medium, low, high_id, medium_id, low_id
['high1', 'med1', 'low1', 1, 1, 1]
['high1', 'med1', 'low1', 1, 1, 1]
['high1', 'med1', 'low2', 1, 1, 2]
['high1', 'med1', 'low3', 1, 1, 3]
['high1', 'med1', 'low3', 1, 1, 3]
['high1', 'med1', 'low3', 1, 1, 3]
['high1', 'med1', 'low4', 1, 1, 4]
['high1', 'med2', 'low5', 1, 2, 1] # medium changed; low_id reset
['high1', 'med2', 'low6', 1, 2, 2]
['high1', 'med3', 'low7', 1, 3, 1] # medium changed; low_id reset
['high1', 'med3', 'low7', 1, 3, 1]
['high1', 'med3', 'low7', 1, 3, 1]
['high1', 'med4', 'low8', 1, 4, 1] # medium changed; low_id reset
['high2', 'med5', 'low9', 2, 1, 1] # high changed; low_id, medium_id reset
['high2', 'med5', 'lowA', 2, 1, 2]
['high2', 'med5', 'lowA', 2, 1, 2]
['high2', 'med6', 'lowB', 2, 2, 1] # medium changed; low_id reset
['high3', 'med4', 'lowC', 3, 1, 1] # high changed; low_id, medium_id reset
['high3', 'med7', 'low1', 3, 2, 1] # medium changed; low_id reset
['high3', 'med7', 'lowD', 3, 2, 2]
['high3', 'med7', 'lowE', 3, 2, 3]

请注意，这些列实际上由地理地名组成：因此，

中

和

低

的值原则上可能会在不同的父级序列中重新出现。（很少有“高”值，我可以看到它们都没有重复。）

添加这些列的惯用方法是什么，最好是通过矢量化操作

---

我已经阅读了许多关于“层次结构”、“计数器”、“标识符”等主题的现有问题，但找不到任何与需要“重置”标识符的特定嵌套案例相匹配的问题。

我不知道这是否是一种常用方法，但我们要求提供将他们分组所需的信息，以便确定他们各自的ID。逻辑是将它们组合在一起，与列表匹配的索引是ID信息。但是，我找不到避免循环处理的方法，所以我使用了循环处理。这可能不会让您满意，但我将作为一种方法来回答

import pandas as pd
import numpy as np
import io

rows = [
    ["high1", "med1", "low1"],
    ["high1", "med1", "low1"],
    ["high1", "med1", "low2"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low3"],
    ["high1", "med1", "low4"],
    ["high1", "med2", "low5"],
    ["high1", "med2", "low6"],
    ["high1", "med3", "low7"],
    ["high1", "med3", "low7"],
    ["high1", "med3", "low7"],
    ["high1", "med4", "low8"],
    ["high2", "med5", "low9"],
    ["high2", "med5", "lowA"],
    ["high2", "med5", "lowA"],
    ["high2", "med6", "lowB"],
    ["high3", "med4", "lowC"],
    ["high3", "med7", "low1"],
    ["high3", "med7", "lowD"],
    ["high3", "med7", "lowE"]]

df = pd.DataFrame(rows, columns=['high','medium','low'])
df['high_id'] = df['high'].str.extract(r'(\d)')
m = df.groupby('high')['medium'].unique().to_frame().reset_index()
l = df.groupby(['high','medium'])['low'].unique().to_frame().reset_index()
df = df.merge(m, on='high', how='outer')
df.rename(columns={'medium_x':'medium'}, inplace=True)
df = df.merge(l, on=['high','medium'], how='outer')

df.tail()
    high    medium  low_x   high_id medium_y    low_y
16  high2   med6    lowB    2   [med5, med6]    [lowB]
17  high3   med4    lowC    3   [med4, med7]    [lowC]
18  high3   med7    low1    3   [med4, med7]    [low1, lowD, lowE]
19  high3   med7    lowD    3   [med4, med7]    [low1, lowD, lowE]
20  high3   med7    lowE    3   [med4, med7]    [low1, lowD, lowE]

df['medium_id'] = ''
for i in range(len(df)):
    con = np.where(df.loc[i,'medium'] == df.loc[i,'medium_y'])
    df.loc[i,'medium_id'] = int(con[0]) + 1

df['low_id'] = ''
for i in range(len(df)):
    con = np.where(df.loc[i,'low_x'] == df.loc[i,'low_y'])
    df.loc[i,'low_id'] = int(con[0]) + 1

df = df[['high', 'medium', 'low_x', 'high_id', 'medium_id','low_id']]
df.columns = ['high', 'medium', 'low', 'high_id', 'medium_id','low_id']
df
    high    medium  low high_id medium_id   low_id
0   high1   med1    low1    1   1   1
1   high1   med1    low1    1   1   1
2   high1   med1    low2    1   1   2
3   high1   med1    low3    1   1   3
4   high1   med1    low3    1   1   3
5   high1   med1    low3    1   1   3
6   high1   med1    low4    1   1   4
7   high1   med2    low5    1   2   1
8   high1   med2    low6    1   2   2
9   high1   med3    low7    1   3   1
10  high1   med3    low7    1   3   1
11  high1   med3    low7    1   3   1
12  high1   med4    low8    1   4   1
13  high2   med5    low9    2   1   1
14  high2   med5    lowA    2   1   2
15  high2   med5    lowA    2   1   2
16  high2   med6    lowB    2   2   1
17  high3   med4    lowC    3   1   1
18  high3   med7    low1    3   2   1
19  high3   med7    lowD    3   2   2
20  high3   med7    lowE    3   2   3

---

这是正确的行为，所以我接受了这个答案。在大型数据集上，它的速度很慢，这是可以理解的，因为它在行之间循环。如果有人提出可行的矢量化替代方案，那么这应该是公认的答案。