Python 偏执狂，在处理文件的简单脚本上过度记录和异常处理。这正常吗？

我发现自己在很多文件管理脚本中使用python，如下所示。在网上寻找示例时，我惊讶地发现示例中很少有日志记录和异常处理。每次我写一个新的脚本，我的目的不是要以下面的脚本结束，但是如果它处理文件，那么不管我的偏执狂是什么，最终的结果与我在网上看到的例子完全不同。作为一个新手，我想知道这是否正常。如果没有，那么你如何处理这些未知因素以及删除有价值信息的恐惧

def flatten_dir(dirname):
    '''Flattens a given root directory by moving all files from its sub-directories and nested 
    sub-directories into the root directory and then deletes all sub-directories and nested 
    sub-directories. Creates a backup directory preserving the original structure of the root
    directory and restores this in case of errors.
    '''
    RESTORE_BACKUP = False
    log.info('processing directory "%s"' % dirname)
    backup_dirname = str(uuid.uuid4())
    try:
        shutil.copytree(dirname, backup_dirname)
        log.debug('directory "%s" backed up as directory "%s"' % (dirname,backup_dirname))
    except shutil.Error:
        log.error('shutil.Error: Error while trying to back up the directory')
        sys.stderr.write('the program is terminating with an error\n')
        sys.stderr.write('press consult the log file\n')
        sys.stderr.flush()
        time.sleep(0.25)
        print 'Press any key to quit this program.'
        msvcrt.getch()
        sys.exit()

    for root, dirs, files in os.walk(dirname, topdown=False):
        log.debug('os.walk passing: (%s, %s, %s)' % (root, dirs, files))
        if root != dirname:
            for file in files:
                full_filename = os.path.join(root, file)
                try:
                    shutil.move(full_filename, dirname)
                    log.debug('"%s" copied to directory "%s"' % (file,dirname))
                except shutil.Error:
                    RESTORE_BACKUP = True
                    log.error('file "%s" could not be copied to directory "%s"' % (file,dirname))
                    log.error('flagging directory "%s" for reset' % dirname)
            if not RESTORE_BACKUP:
                try:
                    shutil.rmtree(root)
                    log.debug('directory "%s" deleted' % root)
                except shutil.Error:
                    RESTORE_BACKUP = True
                    log.error('directory "%s" could not be deleted' % root)
                    log.error('flagging directory "%s" for reset' % dirname)
        if RESTORE_BACKUP:
            break
    if RESTORE_BACKUP:
        RESTORE_FAIL = False
        try:
            shutil.rmtree(dirname)
        except shutil.Error:
            log.error('modified directory "%s" could not be deleted' % dirname)
            log.error('manual restoration from backup directory "%s" necessary' % backup_dirname)
            RESTORE_FAIL = True 
        if not RESTORE_FAIL:
            try:
                os.renames(backup_dirname, dirname)
                log.debug('back up of directory "%s" restored' % dirname)
                print '>'
                print '>******WARNING******'
                print '>There was an error while trying to flatten directory "%s"' % dirname
                print '>back up of directory "%s" restored' % dirname
                print '>******WARNING******'
                print '>'
            except WindowsError:
                log.error('backup directory "%s" could not be renamed to original directory name' % backup_dirname)
                log.error('manual renaming of backup directory "%s" to original directory name "%s" necessary' % (backup_dirname,dirname))
                print '>'
                print '>******WARNING******'
                print '>There was an error while trying to flatten directory "%s"' % dirname
                print '>back up of directory "%s" was NOT restored successfully' % dirname
                print '>no information is lost'
                print '>check the log file for information on manually restoring the directory'
                print '>******WARNING******'
                print '>'
    else:
        try:
            shutil.rmtree(backup_dirname)
            log.debug('back up of directory "%s" deleted' % dirname)
            log.info('directory "%s" successfully processed' % dirname)
            print '>directory "%s" successfully processed' % dirname
        except shutil.Error:
            log.error('backup directory "%s" could not be deleted' % backup_dirname)
            log.error('manual deletion of backup directory "%s" necessary' % backup_dirname)
            print '>'
            print '>******WARNING******'
            print '>directory "%s" successfully processed' % dirname
            print '>cleanup of backup directory "%s" failed' % backup_dirname
            print '>manual cleanup necessary'
            print '>******WARNING******'
            print '>'

---

这对我来说似乎是合理的。这取决于数据的重要性

我通常是这样开始的，并且日志记录是可选的，在文件顶部设置一个标志（或由调用者设置）来设置日志记录的打开或关闭。你也可以说得冗长

---

一般来说，在某些东西已经工作了一段时间并且不再处于开发阶段之后，我会停止阅读日志，并建立我从未阅读过的巨大日志文件。然而，如果确实出了问题，知道他们在那里是很好的。

有点偏执也没关系。但有不同种类的偏执：）。在开发阶段，我使用了很多调试语句，这样我就可以看到哪里出了问题（如果出了问题）。有时我会保留这些语句，但使用一个标志来控制是否需要显示它们（相当于一个调试标志）。您还可以使用一个“详细”标志来控制日志记录量

另一种类型的妄想症伴随着精神检查。当你依赖外部数据或工具时，这种偏执就开始起作用了——几乎所有的东西都不是从你的程序中产生的。在这种情况下，偏执从来都没有坏处（特别是对于你收到的数据-永远不要相信它）

如果你正在检查某个特定的操作是否成功完成，那么你也可以变得偏执。这只是正常错误处理的一部分。我注意到您正在执行删除目录和文件之类的功能。这些操作可能会失败，因此您必须处理它们失败的场景。如果忽略它，代码可能会处于不确定/未定义的状态，并且可能会做不好的事情（或者至少是不希望的事情）

就日志文件和调试文件而言，如果愿意，可以将它们保留在其中。我通常会进行大量的日志记录；足以告诉我发生了什么。当然，这是主观的。关键是确保你不会在伐木中溺水；那里有太多的信息，你不能轻易地把它挑出来。一般来说，当您编写的脚本突然停止工作时，日志记录可以帮助您找出错误所在。通过浏览日志，你可以大致了解问题所在，而不是通过程序来找出问题所在。

学会放手（或者我是如何学会与炸弹共处的）

问问你自己：你到底害怕什么，如果它发生了，你会如何处理？在您提供的示例中，您希望避免数据丢失。您处理它的方法是查找您认为是错误的每一种情况组合，并在其上放置大量日志记录。事情仍然会出问题，现在还不清楚大量的日志记录是否是解决问题的好方法。勾勒出你想要实现的目标：

for each file in a tree
  if file is below the root
    move it into the root
if nothing went wrong
  delete empty subtrees

那么在这个过程中会出现什么样的问题呢？嗯，由于底层文件系统的原因，移动文件操作可能会以多种方式出现问题。我们能把它们全部列出并提供处理它们的好方法吗？不但总的来说，你将以同样的方式处理它们。有时，错误只是一个错误，不管它是什么

因此，在这种情况下，如果发生任何错误，那么您希望中止并撤消任何更改。您决定这样做的方式是创建一个备份副本，并在出现问题时恢复它。但您最可能的错误是文件系统已满，在这种情况下，这些步骤可能会失败。。。。好的，这是一个很常见的问题-如果您担心在任何时候出现未知错误，如何阻止恢复路径出错

一般的答案是，确保您先做任何中间工作，然后采取一个麻烦的步骤（希望是原子步骤）。在你的情况下，你需要扭转你的恢复。生成结果的副本，而不是生成副本作为备份。如果一切都成功，那么可以将新结果替换为旧的原始树。或者，如果你真的是偏执狂，你可以把这一步留给一个人。这里的优点是，如果出现问题，您可以中止并丢弃已构建的部分状态

然后，您的结构将变为：

make empty result directory
for every file in the tree
  copy file into new result
on failure abort otherwise
  move result over old source directory

顺便说一句，在您当前的脚本中有一个bug，这段psuedo代码使之更加明显：如果您在不同的分支中有同名的文件，它们将在新版本中相互覆盖

关于这个psuedo代码的第二点是，所有的错误处理都在同一个位置（即将make new directory和recursive copy包装在一个try块中，并捕获它之后的所有错误），这就解决了日志/错误检查与实际工作代码的比例过大的原始问题

backup_dirname = str(uuid.uuid4())
try:
    shutil.mkdir(backup_dirname)
    for root, dirs, files in os.walk(dirname, topdown=False):
        for file in files:
            full_filename = os.path.join(root, file)
            target_filename = os.path.join(backup_dirname,file)
            shutil.copy(full_filename, target_filename)
catch Exception, e:
    print >>sys.stderr, "Something went wrong %s" % e
    exit(-1)
shutil.move(back_dirname,root)      # I would do this bit by hand really

---

偏执狂肯定会模糊代码的意图。出于几个原因，这是一件非常糟糕的事情。它隐藏着虫子。当你需要它做其他事情时，它会使程序更难修改。这使得调试变得更加困难

假设阿莫斯不能治愈你的偏执狂，下面是我如何重写这个程序。请注意：

• 包含大量偏执狂的每一段代码都被分解成自己的功能

• 每次捕获到异常时，它都是re-r

  def backup_tree(dirname, backup_dirname):
      try:
          shutil.copytree(dirname, backup_dirname)
          log.debug('directory "%s" backed up as directory "%s"' % (dirname,backup_dirname))
      except:
          log.error('Error trying to back up the directory')
          raise
  
  def move_file(full_filename, dirname):
      try:
          shutil.move(full_filename, dirname)
          log.debug('"%s" copied to directory "%s"' % (file,dirname))
      except:
          log.error('file "%s" could not be moved to directory "%s"' % (file,dirname))
          raise
  
  def remove_empty_dir(dirname):
      try:
          os.rmdir(dirname)
          log.debug('directory "%s" deleted' % dirname)
      except:
          log.error('directory "%s" could not be deleted' % dirname)
          raise
  
  def remove_tree_for_restore(dirname):
      try:
          shutil.rmtree(dirname)
      except:
          log.error('modified directory "%s" could not be deleted' % dirname)
          log.error('manual restoration from backup directory "%s" necessary' % backup_dirname)
          raise
  
  def restore_backup(backup_dirname, dirname):
      try:
          os.renames(backup_dirname, dirname)
          log.debug('back up of directory "%s" restored' % dirname)
          print '>'
          print '>******WARNING******'
          print '>There was an error while trying to flatten directory "%s"' % dirname
          print '>back up of directory "%s" restored' % dirname
          print '>******WARNING******'
          print '>'
      except:
          log.error('backup directory "%s" could not be renamed to original directory name' % backup_dirname)
          log.error('manual renaming of backup directory "%s" to original directory name "%s" necessary' % (backup_dirname,dirname))
          print '>'
          print '>******WARNING******'
          print '>There was an error while trying to flatten directory "%s"' % dirname
          print '>back up of directory "%s" was NOT restored successfully' % dirname
          print '>no information is lost'
          print '>check the log file for information on manually restoring the directory'
          print '>******WARNING******'
          print '>'
          raise
  
  def remove_backup_tree(backup_dirname):
      try:
          shutil.rmtree(backup_dirname)
          log.debug('back up of directory "%s" deleted' % dirname)
          log.info('directory "%s" successfully processed' % dirname)
          print '>directory "%s" successfully processed' % dirname
      except shutil.Error:
          log.error('backup directory "%s" could not be deleted' % backup_dirname)
          log.error('manual deletion of backup directory "%s" necessary' % backup_dirname)
          print '>'
          print '>******WARNING******'
          print '>directory "%s" successfully processed' % dirname
          print '>cleanup of backup directory "%s" failed' % backup_dirname
          print '>manual cleanup necessary'
          print '>******WARNING******'
          print '>'
          raise
  
  def flatten_dir(dirname):
      '''Flattens a given root directory by moving all files from its sub-directories and nested 
      sub-directories into the root directory and then deletes all sub-directories and nested 
      sub-directories. Creates a backup directory preserving the original structure of the root
      directory and restores this in case of errors.
      '''
      log.info('processing directory "%s"' % dirname)
      backup_dirname = str(uuid.uuid4())
      backup_tree(dirname, backup_dirname)
      try:
          for root, dirs, files in os.walk(dirname, topdown=False):
              log.debug('os.walk passing: (%s, %s, %s)' % (root, dirs, files))
              if root != dirname:
                  for file in files:
                      full_filename = os.path.join(root, file)
                      move_file(full_filename, dirname)
                  remove_empty_dir(dirname)
      except:
          remove_tree_for_restore(dirname)
          restore_backup(backup_dirname, dirname)
          raise
      else:
          remove_backup_tree(backup_dirname)
  
  def main(dirname):
      try:
          flatten_dir(dirname)
      except:
          import exceptions
          logging.exception('error flattening directory "%s"' % dirname)
          exceptions.print_exc()
          sys.stderr.write('the program is terminating with an error\n')
          sys.stderr.write('press consult the log file\n')
          sys.stderr.flush()
          time.sleep(0.25)
          print 'Press any key to quit this program.'
          msvcrt.getch()
          sys.exit()
  

  import os, logging
  
  def flatten_dir(dirname):
      for root, dirs, files in os.walk(dirname, topdown=False):
          assert len(dirs) == 0
          if root != dirname:
              for file in files:
                  full_filename = os.path.join(root, file)
                  target_filename = os.path.join(dirname, file)
                  if os.path.exists(target_filename):
                      raise Exception('Unable to move file "%s" because "%s" already exists'
                                      % (full_filename, target_filename))
                  os.rename(full_filename, target_filename)
              os.rmdir(root)
  
  def main():
      try:
          flatten_dir(somedir)
      except:
          logging.exception('Failed to flatten directory "%s".' % somedir)
          print "ERROR: Failed to flatten directory. Check log files for details."