Python “Django测试”；“数据库表已锁定”；

在我的Django项目中，我认为当用户发布一个zip文件时，它会立即响应，然后在后台借助线程处理数据。该视图在正常测试中工作正常，但是当我运行Django的测试时，它失败了，数据库表被锁定了。目前，我正在使用默认的SQLite数据库，我知道如果我切换到另一个数据库，这个问题可能会得到解决，但我正在寻找当前设置的答案。为了简单起见，我修改了代码

似乎问题在于写入

DeviceReportModel

表。但是我不确定为什么

TestDeviceReport

访问它

Model.py

：

class DeviceReportModel(models.Model):
    device_id = models.PositiveIntegerField(primary_key=True)
    ip = models.GenericIPAddressField()
    created_time = models.DateTimeField(default=timezone.now)
    report_file = models.FileField(upload_to="DeviceReport")
    device_datas = models.ManyToManyField(DeviceDataReportModel)

    def __str__(self):
        return str(self.id)

class DeviceReportSerializer(serializers.ModelSerializer):
    class Meta:
        model = DeviceReportModel
        fields = '__all__'
        read_only_fields = ('created_time', 'ip', 'device_datas')

from django.utils import timezone
from django.core.files.base import ContentFile
from rest_framework.response import Response
from rest_framework import status, generics
import time
import threading
from queue import Queue

class DeviceReportHandler:
    ReportQueue = Queue()

    @staticmethod
    def save_datas(device_object, request_ip, b64datas):
        device_data_models = []
        # ...
        # process device_data_models
        # this will take some time
        time.sleep(10)
        return device_data_models

    @classmethod
    def Check(cls):
        while(True):
            if not cls.ReportQueue.empty():
                report = cls.ReportQueue.get()
                # ...
                report_model = DeviceReportModel(
                    device_id=report['device_object'], ip=report['request_ip'])
                # THIS LINE GIVES ERROR
                report_model.report_file.save(
                    "Report_{}.txt.gz".format(timezone.now()), ContentFile(report['report_data']))

                device_data_models = cls.save_datas(
                    report['device_object'], report['request_ip'], 'SomeData')
                report_model.device_datas.set(device_data_models)
                report_model.save()
                print("Report Handle Done")
            time.sleep(.1)

    @classmethod
    def run(cls):
        thr = threading.Thread(target=cls.Check)
        thr.daemon = True
        thr.start()


class DeviceReportView(generics.ListCreateAPIView):
    queryset = DeviceReportModel.objects.all()
    serializer_class = DeviceReportSerializer
    DeviceReportHandler.run()

    def post(self, request):
        # ...
        report = {
            'device_object': 1,
            'request_ip': '0.0.0.0',
            'report_data': b'Some report plain data',
        }
        # add request to ReportQueue
        DeviceReportHandler.ReportQueue.put(report)
        return Response("OK", status.HTTP_201_CREATED)

from rest_framework.test import APITestCase
import gzip
from io import BytesIO
import base64
import time

class TestDeviceReport(APITestCase):
    @classmethod
    def setUpTestData(cls):
        # add a new test device for other tests
        pass

    def generate_device_data(self):
        # generate fake device data
        return ""

    def test_Report(self):
        # generate device data
        device_data = ''
        for i in range(10):
            device_data += self.generate_device_data() + '\n'

        buf = BytesIO()
        compressed = gzip.GzipFile(fileobj=buf, mode="wb")
        compressed.write(device_data.encode())
        compressed.close()
        b64data = base64.b64encode(buf.getvalue()).decode()
        data = {
            "device_id": 1,
            "report_data": b64data
        }
        response = self.client.post(
            '/device/reports/', data=data, format='json')
        print(response.status_code, response.content)

    def tearDown(self):
        # put some sleep to check whether the data has been processed
        # see "Report Handle Done"
        time.sleep(10)

Serializers.py

：

class DeviceReportModel(models.Model):
    device_id = models.PositiveIntegerField(primary_key=True)
    ip = models.GenericIPAddressField()
    created_time = models.DateTimeField(default=timezone.now)
    report_file = models.FileField(upload_to="DeviceReport")
    device_datas = models.ManyToManyField(DeviceDataReportModel)

    def __str__(self):
        return str(self.id)

class DeviceReportSerializer(serializers.ModelSerializer):
    class Meta:
        model = DeviceReportModel
        fields = '__all__'
        read_only_fields = ('created_time', 'ip', 'device_datas')

from django.utils import timezone
from django.core.files.base import ContentFile
from rest_framework.response import Response
from rest_framework import status, generics
import time
import threading
from queue import Queue

class DeviceReportHandler:
    ReportQueue = Queue()

    @staticmethod
    def save_datas(device_object, request_ip, b64datas):
        device_data_models = []
        # ...
        # process device_data_models
        # this will take some time
        time.sleep(10)
        return device_data_models

    @classmethod
    def Check(cls):
        while(True):
            if not cls.ReportQueue.empty():
                report = cls.ReportQueue.get()
                # ...
                report_model = DeviceReportModel(
                    device_id=report['device_object'], ip=report['request_ip'])
                # THIS LINE GIVES ERROR
                report_model.report_file.save(
                    "Report_{}.txt.gz".format(timezone.now()), ContentFile(report['report_data']))

                device_data_models = cls.save_datas(
                    report['device_object'], report['request_ip'], 'SomeData')
                report_model.device_datas.set(device_data_models)
                report_model.save()
                print("Report Handle Done")
            time.sleep(.1)

    @classmethod
    def run(cls):
        thr = threading.Thread(target=cls.Check)
        thr.daemon = True
        thr.start()


class DeviceReportView(generics.ListCreateAPIView):
    queryset = DeviceReportModel.objects.all()
    serializer_class = DeviceReportSerializer
    DeviceReportHandler.run()

    def post(self, request):
        # ...
        report = {
            'device_object': 1,
            'request_ip': '0.0.0.0',
            'report_data': b'Some report plain data',
        }
        # add request to ReportQueue
        DeviceReportHandler.ReportQueue.put(report)
        return Response("OK", status.HTTP_201_CREATED)

from rest_framework.test import APITestCase
import gzip
from io import BytesIO
import base64
import time

class TestDeviceReport(APITestCase):
    @classmethod
    def setUpTestData(cls):
        # add a new test device for other tests
        pass

    def generate_device_data(self):
        # generate fake device data
        return ""

    def test_Report(self):
        # generate device data
        device_data = ''
        for i in range(10):
            device_data += self.generate_device_data() + '\n'

        buf = BytesIO()
        compressed = gzip.GzipFile(fileobj=buf, mode="wb")
        compressed.write(device_data.encode())
        compressed.close()
        b64data = base64.b64encode(buf.getvalue()).decode()
        data = {
            "device_id": 1,
            "report_data": b64data
        }
        response = self.client.post(
            '/device/reports/', data=data, format='json')
        print(response.status_code, response.content)

    def tearDown(self):
        # put some sleep to check whether the data has been processed
        # see "Report Handle Done"
        time.sleep(10)

views.py

：

class DeviceReportModel(models.Model):
    device_id = models.PositiveIntegerField(primary_key=True)
    ip = models.GenericIPAddressField()
    created_time = models.DateTimeField(default=timezone.now)
    report_file = models.FileField(upload_to="DeviceReport")
    device_datas = models.ManyToManyField(DeviceDataReportModel)

    def __str__(self):
        return str(self.id)

class DeviceReportSerializer(serializers.ModelSerializer):
    class Meta:
        model = DeviceReportModel
        fields = '__all__'
        read_only_fields = ('created_time', 'ip', 'device_datas')

from django.utils import timezone
from django.core.files.base import ContentFile
from rest_framework.response import Response
from rest_framework import status, generics
import time
import threading
from queue import Queue

class DeviceReportHandler:
    ReportQueue = Queue()

    @staticmethod
    def save_datas(device_object, request_ip, b64datas):
        device_data_models = []
        # ...
        # process device_data_models
        # this will take some time
        time.sleep(10)
        return device_data_models

    @classmethod
    def Check(cls):
        while(True):
            if not cls.ReportQueue.empty():
                report = cls.ReportQueue.get()
                # ...
                report_model = DeviceReportModel(
                    device_id=report['device_object'], ip=report['request_ip'])
                # THIS LINE GIVES ERROR
                report_model.report_file.save(
                    "Report_{}.txt.gz".format(timezone.now()), ContentFile(report['report_data']))

                device_data_models = cls.save_datas(
                    report['device_object'], report['request_ip'], 'SomeData')
                report_model.device_datas.set(device_data_models)
                report_model.save()
                print("Report Handle Done")
            time.sleep(.1)

    @classmethod
    def run(cls):
        thr = threading.Thread(target=cls.Check)
        thr.daemon = True
        thr.start()


class DeviceReportView(generics.ListCreateAPIView):
    queryset = DeviceReportModel.objects.all()
    serializer_class = DeviceReportSerializer
    DeviceReportHandler.run()

    def post(self, request):
        # ...
        report = {
            'device_object': 1,
            'request_ip': '0.0.0.0',
            'report_data': b'Some report plain data',
        }
        # add request to ReportQueue
        DeviceReportHandler.ReportQueue.put(report)
        return Response("OK", status.HTTP_201_CREATED)

from rest_framework.test import APITestCase
import gzip
from io import BytesIO
import base64
import time

class TestDeviceReport(APITestCase):
    @classmethod
    def setUpTestData(cls):
        # add a new test device for other tests
        pass

    def generate_device_data(self):
        # generate fake device data
        return ""

    def test_Report(self):
        # generate device data
        device_data = ''
        for i in range(10):
            device_data += self.generate_device_data() + '\n'

        buf = BytesIO()
        compressed = gzip.GzipFile(fileobj=buf, mode="wb")
        compressed.write(device_data.encode())
        compressed.close()
        b64data = base64.b64encode(buf.getvalue()).decode()
        data = {
            "device_id": 1,
            "report_data": b64data
        }
        response = self.client.post(
            '/device/reports/', data=data, format='json')
        print(response.status_code, response.content)

    def tearDown(self):
        # put some sleep to check whether the data has been processed
        # see "Report Handle Done"
        time.sleep(10)

tests.py

：

class DeviceReportModel(models.Model):
    device_id = models.PositiveIntegerField(primary_key=True)
    ip = models.GenericIPAddressField()
    created_time = models.DateTimeField(default=timezone.now)
    report_file = models.FileField(upload_to="DeviceReport")
    device_datas = models.ManyToManyField(DeviceDataReportModel)

    def __str__(self):
        return str(self.id)

class DeviceReportSerializer(serializers.ModelSerializer):
    class Meta:
        model = DeviceReportModel
        fields = '__all__'
        read_only_fields = ('created_time', 'ip', 'device_datas')

from django.utils import timezone
from django.core.files.base import ContentFile
from rest_framework.response import Response
from rest_framework import status, generics
import time
import threading
from queue import Queue

class DeviceReportHandler:
    ReportQueue = Queue()

    @staticmethod
    def save_datas(device_object, request_ip, b64datas):
        device_data_models = []
        # ...
        # process device_data_models
        # this will take some time
        time.sleep(10)
        return device_data_models

    @classmethod
    def Check(cls):
        while(True):
            if not cls.ReportQueue.empty():
                report = cls.ReportQueue.get()
                # ...
                report_model = DeviceReportModel(
                    device_id=report['device_object'], ip=report['request_ip'])
                # THIS LINE GIVES ERROR
                report_model.report_file.save(
                    "Report_{}.txt.gz".format(timezone.now()), ContentFile(report['report_data']))

                device_data_models = cls.save_datas(
                    report['device_object'], report['request_ip'], 'SomeData')
                report_model.device_datas.set(device_data_models)
                report_model.save()
                print("Report Handle Done")
            time.sleep(.1)

    @classmethod
    def run(cls):
        thr = threading.Thread(target=cls.Check)
        thr.daemon = True
        thr.start()


class DeviceReportView(generics.ListCreateAPIView):
    queryset = DeviceReportModel.objects.all()
    serializer_class = DeviceReportSerializer
    DeviceReportHandler.run()

    def post(self, request):
        # ...
        report = {
            'device_object': 1,
            'request_ip': '0.0.0.0',
            'report_data': b'Some report plain data',
        }
        # add request to ReportQueue
        DeviceReportHandler.ReportQueue.put(report)
        return Response("OK", status.HTTP_201_CREATED)

from rest_framework.test import APITestCase
import gzip
from io import BytesIO
import base64
import time

class TestDeviceReport(APITestCase):
    @classmethod
    def setUpTestData(cls):
        # add a new test device for other tests
        pass

    def generate_device_data(self):
        # generate fake device data
        return ""

    def test_Report(self):
        # generate device data
        device_data = ''
        for i in range(10):
            device_data += self.generate_device_data() + '\n'

        buf = BytesIO()
        compressed = gzip.GzipFile(fileobj=buf, mode="wb")
        compressed.write(device_data.encode())
        compressed.close()
        b64data = base64.b64encode(buf.getvalue()).decode()
        data = {
            "device_id": 1,
            "report_data": b64data
        }
        response = self.client.post(
            '/device/reports/', data=data, format='json')
        print(response.status_code, response.content)

    def tearDown(self):
        # put some sleep to check whether the data has been processed
        # see "Report Handle Done"
        time.sleep(10)

下面是错误日志：

(myDjangoEnv) python manage.py test deviceApp.tests.tests.TestDeviceReport
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
201 b'"OK"'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute       
    return self.cursor.execute(sql, params)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\sqlite3\base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: database table is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "<project_path>\deviceApp\views.py", line 303, in Check
    "Report_{}.txt.gz".format(timezone.now()), ContentFile(report['report_data']))
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\fields\files.py", line 93, in save
    self.instance.save()
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\base.py", line 741, in save
    force_update=force_update, update_fields=update_fields)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\base.py", line 779, in save_base
    force_update, using, update_fields,
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\base.py", line 870, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\base.py", line 908, in _do_insert
    using=using, raw=raw)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\models\sql\compiler.py", line 1335, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site-packages\django\db\backends\sqlite3\base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database table is locked

.
----------------------------------------------------------------------
Ran 1 test in 10.023s

OK
Destroying test database for alias 'default'...

（mydjangenv）python manage.py测试设备app.tests.tests.TestDeviceReport
正在为别名“default”创建测试数据库。。。
系统检查未发现任何问题（0静音）。
201 b“好的”
线程1中的异常：
回溯（最近一次呼叫最后一次）：
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\utils.py”，第84行，在执行
返回self.cursor.execute（sql，params）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\sqlite3\base.py”，执行中第383行
返回Database.Cursor.execute（self、query、params）
sqlite3.0错误：数据库表已锁定
上述异常是以下异常的直接原因：
回溯（最近一次呼叫最后一次）：
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\threading.py”，第917行，在\u bootstrap\u inner中
self.run（）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\threading.py”，第865行，正在运行
自我目标（*自我参数，**自我参数）
文件“\deviceApp\views.py”，第303行，处于选中状态
格式（timezone.now（）），内容文件（Report['Report\u data']））
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\fields\files.py”，第93行，保存
self.instance.save（）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\base.py”，第741行，保存
强制更新=强制更新，更新字段=更新字段）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\base.py”，第779行，在save\u base中
强制更新，使用，更新字段，
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\base.py”，第870行，在保存表中
结果=self.\u do\u insert（cls.\u base\u manager，using，fields，update\u pk，raw）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\base.py”，第908行，插入
使用=使用，原始=原始）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\manager.py”，第82行，在manager\u方法中
返回getattr（self.get_queryset（），name）（*args，**kwargs）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\query.py”，第1186行，插入
return query.get\u编译器（using=using）.execute\u sql（return\u id）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\models\sql\compiler.py”，第1335行，在execute\U sql中
cursor.execute（sql，params）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\utils.py”，执行中第67行
返回self.\u使用包装器执行（sql，params，many=False，executor=self.\u execute）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\utils.py”，第76行，带包装器执行
返回执行器（sql、参数、多个、上下文）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\utils.py”，第84行，在执行
返回self.cursor.execute（sql，params）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\utils.py”，第89行，在退出时__
使用exc_值的_回溯（回溯）提高dj_exc_值
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\utils.py”，第84行，在执行
返回self.cursor.execute（sql，params）
文件“C:\Users\Masoud\Anaconda3\envs\myDjangoEnv\lib\site packages\django\db\backends\sqlite3\base.py”，执行中第383行
返回Database.Cursor.execute（self、query、params）
django.db.utils.OperationalError:数据库表已锁定
.
----------------------------------------------------------------------
在10.023s中运行1次测试
好啊
正在销毁别名“default”的测试数据库。。。

数据库已锁定错误

SQLite是一个轻量级数据库，因此不能支持高级别的并发性。OperationalError：数据库被锁定错误表示您的应用程序遇到的并发比sqlite在默认配置中能够处理的还要多。此错误意味着一个线程或进程在数据库连接上具有独占锁，而另一个线程在等待释放锁时超时

Python的SQLite包装器有一个默认超时值，该值确定第二个线程在锁超时并引发OperationalError（操作性错误）：database is locked error（数据库已锁定错误）之前允许在锁上等待多长时间

如果出现此错误，可以通过以下方法解决：

切换到另一个数据库后端。在某一点上，SQLite对于现实世界的应用程序来说变得过于“精简”，而这些并发错误表明您已经达到了这一点

重写代码以降低并发性并确保数据库事务是短期的

通过设置超时数据库选项增加默认超时值：

'OPTIONS': {
    # ...
    'timeout': 20,
    # ...
}

这将使SQLite在抛出“数据库已锁定”错误之前等待更长的时间；它对解决这些问题没有任何作用

---

谢谢，但我已经试过了。看来我的问题