Python 比较数据帧中的行值

我在熊猫数据框中有数据，其中两列包含数字序列（开始和停止）。我想确定哪些行的停止值与下一行的开始值重叠。然后我需要将它们连接到一行中，这样每行中只有一个无重叠的数字序列，由我的开始值和停止值表示

我已将数据加载到熊猫数据框中：

我想比较数据框中的行。检查每行的停止值是否小于下一行的开始值，然后使用正确的开始值和停止值在新数据帧中创建一行。理想情况下，当有多行都重叠时，这将一次连接所有行，但是我怀疑我将不得不迭代我的输出，直到不再发生这种情况

到目前为止，我的代码可以识别是否存在重叠（改编自）：

我想将此中间数据帧放入以下函数中，以获得所需的输出（如下所示）：

期望输出：

但是，当我运行该函数时，我得到的原始数据帧没有改变。我知道问题是当我尝试调用bed.itertuples（）.next（）时，因为这显然不是调用的正确语法/位置。但我不知道正确的方法来纠正这一点

一些指针会很好

（某人：）

使现代化

---

这是一个序列，其中每行引用一个带有起始和终止坐标的扩增子（基因组区域）。一些放大器重叠；即开始坐标在前一行的停止坐标之前。因此，我需要确定哪些行重叠，并连接正确的开始和停止，以便每一行都代表一个完全唯一的放大器，不与任何其他行重叠。

我将尝试给您一些指针

一个指针是，您希望根据由移位的布尔值组成的序列获取行。可能您可以使用以下方法获得新的移位序列：

Boolean_Series = intermediate_bed.loc[:,'next_region_overlap'].shift(periods=1, freq=None, axis=0, **kwds)

有关此功能的更多背景信息：

第二个指针是，通过使用此移位序列，您可以通过以下方式获得数据帧：

int_bed = bed.loc[Boolean_Series, :] 

有关索引的更多信息，请参见：

---

现在这些只是指针，我不知道这是否是一个实际可行的解决方案

我会给你一些建议

一个指针是，您希望根据由移位的布尔值组成的序列获取行。可能您可以使用以下方法获得新的移位序列：

Boolean_Series = intermediate_bed.loc[:,'next_region_overlap'].shift(periods=1, freq=None, axis=0, **kwds)

有关此功能的更多背景信息：

第二个指针是，通过使用此移位序列，您可以通过以下方式获得数据帧：

int_bed = bed.loc[Boolean_Series, :] 

有关索引的更多信息，请参见：

---

现在这些只是指针，我不知道这是否是一个实际可行的解决方案

我不确定我是否理解您为什么要这样做，但您可以通过简单地使用索引来获得所需的输出。e、 g

# assume your data is stored in <df>
# call the temporary dataframe <tmp>
tmp = df[ ['chr','start','stop','geneID'] ][(df.stop - df.start.shift(-1))>0]

---

我不确定我是否理解您为什么要做您正在做的事情，但您可以通过简单地使用索引来获得所需的输出。e、 g

# assume your data is stored in <df>
# call the temporary dataframe <tmp>
tmp = df[ ['chr','start','stop','geneID'] ][(df.stop - df.start.shift(-1))>0]

---

我修改了bed_prepare函数，以检查上一个和下一个基因组区域的重叠：

def bed_prepare(inp_bed):
    ''' Takes pandas dataframe bed file and identifies which regions overlap '''
    inp_bed['next_start'] = inp_bed['start'].shift(periods=-1)
    inp_bed['distance_to_next'] = inp_bed['next_start'] - inp_bed['stop']
    inp_bed['next_region_overlap'] = inp_bed['next_start'] <= inp_bed['stop']
    inp_bed['previous_stop'] = inp_bed['stop'].shift(periods=1)
    inp_bed['distance_from_previous'] = inp_bed['start'] - inp_bed['previous_stop']
    inp_bed['previous_region_overlap'] = inp_bed['previous_stop'] >= inp_bed['start']
    intermediate_bed = inp_bed
    return intermediate_bed

---

这解决了我的问题，并给出了问题中指定的所需输出。：）

我修改了bed_prepare函数，以检查上一个和下一个基因组区域的重叠：

def bed_prepare(inp_bed):
    ''' Takes pandas dataframe bed file and identifies which regions overlap '''
    inp_bed['next_start'] = inp_bed['start'].shift(periods=-1)
    inp_bed['distance_to_next'] = inp_bed['next_start'] - inp_bed['stop']
    inp_bed['next_region_overlap'] = inp_bed['next_start'] <= inp_bed['stop']
    inp_bed['previous_stop'] = inp_bed['stop'].shift(periods=1)
    inp_bed['distance_from_previous'] = inp_bed['start'] - inp_bed['previous_stop']
    inp_bed['previous_region_overlap'] = inp_bed['previous_stop'] >= inp_bed['start']
    intermediate_bed = inp_bed
    return intermediate_bed

这解决了我的问题，并给出了问题中指定的所需输出。：）

将允许您在一行代码中快速完成此操作：

import pyranges as pr

c = """Chromosome     Start       End        geneID
chr13   32889584    32889814    BRCA2
chr13   32890536    32890737    BRCA2
chr13   32893194    32893307    BRCA2
chr13   32893282    32893400    BRCA2
chr13   32893363    32893466    BRCA2
chr13   32899127    32899242    BRCA2"""

gr = pr.from_string(c)
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893307 | BRCA2      |
# | chr13        |  32893282 |  32893400 | BRCA2      |
# | chr13        |  32893363 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 6 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

m = gr.merge(by="geneID")
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 4 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

请注意，

by=“geneID”

使得只有当间隔重叠并且对于

geneID

具有相同的值时，才会合并间隔。如果要将间隔元数据与自定义函数合并，请参见方法群集。

将允许您在一行代码中快速完成此操作：

import pyranges as pr

c = """Chromosome     Start       End        geneID
chr13   32889584    32889814    BRCA2
chr13   32890536    32890737    BRCA2
chr13   32893194    32893307    BRCA2
chr13   32893282    32893400    BRCA2
chr13   32893363    32893466    BRCA2
chr13   32899127    32899242    BRCA2"""

gr = pr.from_string(c)
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893307 | BRCA2      |
# | chr13        |  32893282 |  32893400 | BRCA2      |
# | chr13        |  32893363 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 6 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

m = gr.merge(by="geneID")
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 4 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

---

请注意，

by=“geneID”

使得只有当间隔重叠并且对于

geneID

具有相同的值时，才会合并间隔。如果要将间隔元数据与自定义函数合并，请参见方法群集。

谢谢您的帮助，但这不是我想要的。您的代码提供给我的新数据帧与我的中间数据帧相同。我已经更新了原来的帖子以增加清晰度。希望这会有所帮助。：）谢谢你的帮助，但那不是我想要的。您的代码提供给我的新数据帧与我的中间数据帧相同。我已经更新了原来的帖子以增加清晰度。希望这会有所帮助。：）

>>> df_cut
     chr     start      stop geneID  tdiff
0  chr13  32889584  32889814  BRCA2    722
1  chr13  32890536  32890737  BRCA2   2457
2  chr13  32893194  32893466  BRCA2     -0

def bed_prepare(inp_bed):
    ''' Takes pandas dataframe bed file and identifies which regions overlap '''
    inp_bed['next_start'] = inp_bed['start'].shift(periods=-1)
    inp_bed['distance_to_next'] = inp_bed['next_start'] - inp_bed['stop']
    inp_bed['next_region_overlap'] = inp_bed['next_start'] <= inp_bed['stop']
    inp_bed['previous_stop'] = inp_bed['stop'].shift(periods=1)
    inp_bed['distance_from_previous'] = inp_bed['start'] - inp_bed['previous_stop']
    inp_bed['previous_region_overlap'] = inp_bed['previous_stop'] >= inp_bed['start']
    intermediate_bed = inp_bed
    return intermediate_bed

# Create empty dataframe to fill with parsed values                                                                   
new_bed = pd.DataFrame(data=np.zeros((0,len(columns))),columns=columns,dtype=int)

def bed_collapse(intermediate_bed, new_bed,columns=columns):
    ''' Takes a pandas dataframe bed file with overlap information and returns                                        
    genomic regions without overlaps '''
    output_row = []
    for row in bed.itertuples():
        output = {}
        if row[7] == False and row[10] == False:
            # If row doesn't overlap next row, insert into new dataframe unchanged.                                   
            output_row = list(row[1:5])
        elif row[7] == True and row[10] == False:
            # Only next region overlaps; take the chromosome and start coordinate                                     
            output_row = list(row[1:3])
        elif row[7] == True and row[10] == True:
            # Next and previous regions overlap. Skip row.                                                            
            pass
        elif row[7] == False and row[10]  == True:
            # Only previous region overlaps; append stop coordinate and geneID to output_row variable                 
            output_row.append(row[3])
            output_row.append(row[4])
        if row[7] == False:
            #Zip columns and output_row values together to form a dict for appending                                  
            for k, v in zip(columns,output_row): output[k] = v
            #print output                                                                                             
            new_bed = new_bed.append(output,ignore_index=True)
    output_bed = new_bed
    return output_bed

import pyranges as pr

c = """Chromosome     Start       End        geneID
chr13   32889584    32889814    BRCA2
chr13   32890536    32890737    BRCA2
chr13   32893194    32893307    BRCA2
chr13   32893282    32893400    BRCA2
chr13   32893363    32893466    BRCA2
chr13   32899127    32899242    BRCA2"""

gr = pr.from_string(c)
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893307 | BRCA2      |
# | chr13        |  32893282 |  32893400 | BRCA2      |
# | chr13        |  32893363 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 6 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

m = gr.merge(by="geneID")
# +--------------+-----------+-----------+------------+
# | Chromosome   |     Start |       End | geneID     |
# | (category)   |   (int32) |   (int32) | (object)   |
# |--------------+-----------+-----------+------------|
# | chr13        |  32889584 |  32889814 | BRCA2      |
# | chr13        |  32890536 |  32890737 | BRCA2      |
# | chr13        |  32893194 |  32893466 | BRCA2      |
# | chr13        |  32899127 |  32899242 | BRCA2      |
# +--------------+-----------+-----------+------------+
# Unstranded PyRanges object has 4 rows and 4 columns from 1 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.