Django community: RSS
This page, updated regularly, aggregates Django Q&A from the Django community.
-
Static and media urls in Django
In my settings file there is STATIC_URL = '/static/' and getting access to static files(css files) from the directory app/static/css/filename . If I add this to the setting file MEDIA_URL = '/media/' is it possible to access the media files from app/media/filename? There is no MEDIA_ROOT or STATIC_ROOT in my settings.py. -
Django URL mapping for different apps
Im trying to link a button in HTML to another html in my project folder for my django project. Lets say its MyApp -Polls -templates -index -Votes -templates -main -truths -rigged I have a a button on main that uses rigged and truths so in main it has this button <form action="{% url 'Votes:rigged' %}"> <input type="submit" value="rigged votes" /> </form> now i want to add another button that would link polls->index into it. Is there a way to do that without copying everything from Polls into the folder Votes? -
unit testing django ModelFormset clean methods
What is the best way to unit test the validation/clean part of a Django ModelFormset? My formset has a clean method that does some validation based on the values of its forms and I want to have a unit test for it. Generified code look like this: class AForm(ModelForm): a = ChoiceField(choices=CHOICES) b = FloatField() def __init__(self, *args, **kwargs): super(AForm, self).__init__(*args, **kwargs) class Meta: model = AModel fields =['a', 'b'] class AFormset(BaseInlineFormSet): def clean(self): # Some logic to validate relationships between the forms But while testing the form is trivial: form = AForm(formdata) and then verifying its validity or errors based on the data. I'm having trouble writing tests for the formset. I've tried: formset = AFormset() And using modelformset_factory Formset = modelformset_factory(AModel, AForm, formset=AFormset, fields=('a', 'b')) formset = Formset(formdata) with various combinations of arguments and mocks (instance, queryset). But I always get errors related to model foreign or primary keys: AttributeError: 'AFormset' object has no attribute 'fk' What am I missing? Is there an easier way to instantiate a formset with a dictionary of data and have it run its clean method? Should I just test the view that the form is used in? (In the views I'm using … -
Make key name change dynamically
I need to change the "key" of my data when it outputs. I am unsure if django can do this. FWIW, this outputs to elastic search (not sure that is relevant). Basically it comes out with {"level":1 } but I want it to say {"level_1":1} instead. How can this be achieved? Here is my current code class Tag(models.Model): name = models.CharField("Name", max_length=5000, blank=True) level = models.IntegerField(null=True, blank=True) class Entry(models.Model): title = models.CharField("Title", max_length=10000, blank=True) tag = models.ManyToManyField('Tag', blank=True) def indexing(self): obj = TaskIndex( meta={'id': self.id}, title=self.title, tag=list(self.tag.values('name', 'level') ) obj.save() return obj.to_dict(include_meta=True) How result looks: [ { "title":"Test item", "tag":[ { "name":"Nope", "level":1 }, { "name":"Yep", "level":2 } ], } ] How I want the result to look: [ { "title":"Test item", "tag":[ { "name":"Nope", "level_1":1 }, { "name":"Yep", "level_2":2 } ], } ] -
Form in django form.as_p diffrent
what form.to_p exacly return? I mean this code: <form action="{{ action }}" method="post" enctype="multipart/form-data"> {% csrf_token %} <input type="text" name="title_field" id="form.title}}"/> <input type="text" name="author_field" id="form.author }}"/> {{ form.content }} <input type="submit" value="Send"/> </form> doesn't work, instead of this code works: <form action="" method="post" enctype="multipart/form-data"> {% csrf_token %} {{ form.as_p }} <input type="submit" value="Send"/> </form> Of course in the first case I can stylize in html/css specific fields.. -
Separating dependencies for Celery container from Django container
We currently have a separate Docker container for Django and Celery but they share the same root files. I was wondering if there was a way to install OpenCV in the Celery container, because we use the package in celery tasks, but not in the Django container calling the celery task. When installing OpenCV in the celery container and not the Django container, the Django container throws a "Could not import cv" error from tasks.py. What is the best way to solve this problem? -
Django: prefetch_related() with m2m through relationship v2
I know there is already a similar question, but I think my case is a bit more complicated because I have a different entry point. These are my models: class m_Interaction(models.Model): fk_ip = models.ForeignKey('m_IP', related_name="interactions") class m_User(models.Model): name = models.CharField(max_length=200) class m_IP(models.Model): fk_user = models.ForeignKey('m_User', related_name="ips" ) class m_Feature(models.Model): name = models.CharField(max_length=200) m2m_interaction = models.ManyToManyField(m_Interaction, related_name='features', through='m_Featurescore') class m_Featurescore(models.Model): score = models.FloatField(null=False) fk_interaction = models.ForeignKey(m_Interaction, related_name='featurescore') fk_feature = models.ForeignKey(m_Feature, related_name='featurescore') I start with m_User, follow the reverse relationship over m_IP to the Interactions. Then I want to get every m_Featurescore.score for each Interaction for a specific instance of m_Feature. My working query to access at least all interactions in a performant way: m_User.objects.all().prefetch_related('ips__interactions') But I'm can't figure out the correct 'prefetch_related'-statement to access the m_Featurescore.score like this db_obj_interaction.featurescore.get(fk_feature=db_obj_feature).score without making a lot of queries. Any suggestions? -
Duplicate key error on non-primary key field
I am getting the follow error with django (1062, "Duplicate entry '2' for key 'building_id'") Here is what the model looks like class BuildingProgressComments(models.Model): user = models.ForeignKey(User) building = models.OneToOneField('Building') date_created = models.DateTimeField(auto_now_add=True) comment = models.TextField(blank=True, null=True) class Meta: verbose_name='Building progress notes' def __unicode__(self): return unicode(self.building) I am trying to add multiple entries for a single building and the building field isn't a primary key so why wont this work? Thanks in advance -
Django Authenticate against external SSO
Forgive me if the wording here is a bit odd. I have a fair amount of experience with python, but I'm pretty new to Django and still trying to get my sea legs. I've inherited a PHP site from the former System admin at my new job, and I want to rework it into a django app to simplify/cleanup the code. The PHP site is using .htaccess file with Pubcookie apache module for authentication (as well as some basic checks in the php code to ensure the user checks out against a database of authorized users, but that part i'm less concerned with), and looks like this: AuthType SecureID require valid-user PubcookieAppID "name of the app" In my apache configs, I have various pubcookie definitions. LoadModule pubcookie_module /usr/lib/apache2/modules/mod_pubcookie.so PubcookieGrantingCertFile /usr/local/pubcookie/keys/pubcookie_granting.cert PubcookieSessionKeyFile /etc/ssl/private/akey.pem PubcookieSessionCertFile /etc/ssl/certs/acert.pem PubcookieLogin https://weblogin.domain.com/ PubcookieLoginMethod POST PubcookieDomain .domain.com PubcookieKeyDir /usr/local/pubcookie/keys/ PubcookieAuthTypeNames ADUserID null SecurID My question is...How do I implement this sort of thing on the python end? I have looked at using RemoteUserMiddleware, or PersistentRemoteUserMiddleware. But most of the information I have found for that doesn't go into great detail about how to actually point your app at the correct login page. The official documentation seems to … -
Jinja Tag and variables, Invalid Block Tag: Set Expected End block
I'm trying to set jinja variables in a page contained in my project. Project structure is a base.html file with other html files that extend the base. inside of the {% block content %} tags of my record page. I'm trying to perform some math with variables by using set to name them: <td><input type="number" class="form-control" placeholder="Daily Total" value="{{ object.incoming_meter_2_close|sub:object.incoming_meter_2_open }}"/> </td> <tr> {% set metertotal2 = object.incoming_meter_2_close|sub:object.incoming_meter_2_open %} {% set metertotal1 = object.incoming_meter_1_close|sub:object.incoming_meter_1_open %} <th>Total</th> <td><input type="number" class="form-control" placeholder="Combined Total" value="{{ metertotal1|add:metertotal2 }}"/></td> </tr> When I run my code I'm being given the error: Invalid block tag on line 198: 'set', expected 'endblock'. Did you forget to register or load this tag? The code is contained within the {% block content %} and {% Endblock %} tags for the page. Where do I need to have another end statement? I've tried to add them in a number of places and none of them are working. -
Python Django migrate not picking up change from makemigrations
I added a new model to models.py corresponding to a new SQL table I need to create. I included a field, Id, which apparently it does not like, as python manage.py migrate said that it was a duplicate field (presumably an Id field is also being automatically added or something). That's fine, so I removed that field from the model and re-ran python manage.py makemigrations, which correctly reported that the Id field was being removed. I think re-ran python manage.py migrate, but it still gives the same error of there being a duplicate Id field... From models.py: class TII(models.Model): Person_bnr_id = models.CharField(max_length = 128, unique = True) COURSE_IDENTIFICATION = models.CharField(max_length = 128, unique = True) Course_start = models.CharField(max_length = 128, unique = True) Course_end = models.CharField(max_length = 128, unique = True) TurnItIn_W01 = models.CharField(max_length = 128, unique = True) TurnItIn_W02 = models.CharField(max_length = 128, unique = True) TurnItIn_W03 = models.CharField(max_length = 128, unique = True) TurnItIn_W04 = models.CharField(max_length = 128, unique = True) TurnItIn_W05 = models.CharField(max_length = 128, unique = True) TurnItIn_W06 = models.CharField(max_length = 128, unique = True) TurnItIn_W07 = models.CharField(max_length = 128, unique = True) TurnItIn_W08 = models.CharField(max_length = 128, unique = True) TurnItIn_W09 = models.CharField(max_length = 128, … -
Save method is not triggering on object save
I have a save method on my model which I would like to use to validate the fields (I did this in my views before but it was just messy!). I'm saving an object rather than a form because I was having some difficulties with the form validation. I haven't used save methods before, but I assume they trigger when you use the .save() command? Mine isn't triggering at all... Also from what I understand a clean() method is run for forms when you use is_valid(), so isn't useful for me here? Here is my view that saves an object using ajax @login_required def createtimesheet(request): if request.method == "POST": print "creating timesheet" # Get the person from the previous view person_object = request.session.get('person') person = Person.objects.get(id=person_object) # get the POST data start_date = request.POST.get('datestart') end_date = request.POST.get('dateend') start_date_formatted = datetime.strptime(start_date, "%m/%d/%Y") end_date_formatted = datetime.strptime(end_date, "%m/%d/%Y") start_date_print = start_date_formatted.strftime("%B")[0:3] + ". " + start_date_formatted.strftime("%d") + ", " + start_date_formatted.strftime("%Y") end_date_print = end_date_formatted.strftime("%B")[0:3] + ". " + end_date_formatted.strftime("%d") + ", " + end_date_formatted.strftime("%Y") response_data = {} # create our messages message = "" error = "" todays_date = datetime.now() print_date = todays_date.strftime("%A") # now we can create the timesheet! peach = TimeSheet(start_date=start_date_formatted, … -
class UserAdmin(admin.ModelAdmin) override delete function for multi-delete
I'm using the default User class for the django authentication system. I've overrided the delete_model function on the UserAdmin class as below to execute some stuff when deleting my user. It works fine when I go to the user on the admin page, and then click on the delete button. My problem it's when I seletect one or multiple user on the UserAdmin page, it looks like this is not the delete_model function which is called. What should I do to fix it? from django.contrib import admin #create a custom model admin which allow to search on the email field class UserAdmin(admin.ModelAdmin): #override the delete method to unset the send email to the deleted_user def delete_model(UserAdmin, request, obj): #### do something #### obj.delete() #unregister the basic User, and register the new User ModelAdmin admin.site.unregister(User) admin.site.register(User, UserAdmin) -
Passing Token Using Fetch to a Django Rest API in React Native
I have my rest-api set up in Django and am using React Native to connect with it. I have registered users and am able to generate tokens however I am unable to pass the token in the header of the GET request. My code is as follows: try{ let response = await fetch("http://127.0.0.1:8000/fishes/auth/", { method: 'GET', headers: { // 'Accept': 'application/json', 'Content-Type': 'application/json', 'Authorization': ' Token '+accessToken, }}); let res = await response.text(); }} I have been following this link http://cheng.logdown.com/posts/2015/10/27/how-to-use-django-rest-frameworks-token-based-authentication and have already verified that the response from the rest api is correct. However on the phone with native react I get the following error in the console: TypeError: Network request failed at XMLHttpRequest.xhr.onerror (fetch.js:441) at XMLHttpRequest.dispatchEvent (event-target.js:172) at XMLHttpRequest.setReadyState (XMLHttpRequest.js:542) What am I doing wrong in the GET code? -
RuntimeError maximum recursion depth exceeded while calling a Python object
I use python2.7. django I have a prolblem that message is 'RuntimeError maximum recursion depth exceeded while calling a Python object'. here is my code def save(self, *args, **kwargs): if self.id: # Have to save the image (and imagefield) first super(Profile, self).save(*args, **kwargs) # obj is being created for the first time - resize resized = get_thumbnail(self.profile_image, '100x100', crop='center', quality=99) # Manually reassign the resized image to the image field print("dsjfhkasdhfksdfkjsbdfkajsbdflkjshdflkjhsdklfjhsdkjfb") print(resized) # self.profile_image_thumnail.save(resized.name, resized.read(), True) self.profile_image.save(resized.name, ContentFile(resized.read()), True) else: super(Profile, self).save(*args, **kwargs) and def save(self, name, content, save=True): name = self.field.generate_filename(self.instance, name) self.name = self.storage.save(name, content, max_length=self.field.max_length) setattr(self.instance, self.field.name, self.name) self._committed = True # Save the object because it has changed, unless save is False if save: self.instance.save() save.alters_data = True and my error log is [enter image description here][1] File "/Desktop/project/src/accounts/models.py", line 80, in save self.profile_image.save(resized.name, ContentFile(resized.read()), True) File "/Desktop/project/lib/python2.7/site-packages/django/db/models/fields/files.py", line 97, in save self.instance.save() File "/Desktop/project/src/accounts/models.py", line 80, in save self.profile_image.save(resized.name, ContentFile(resized.read()), True) File "/Desktop/project/lib/python2.7/site-packages/django/db/models/fields/files.py", line 97, in save self.instance.save() File "/Desktop/project/src/accounts/models.py", line 80, in save self.profile_image.save(resized.name, ContentFile(resized.read()), True) File "/Desktop/project/lib/python2.7/site-packages/django/db/models/fields/files.py", line 97, in save self.instance.save() File "/Desktop/project/src/accounts/models.py", line 80, in save self.profile_image.save(resized.name, ContentFile(resized.read()), True) File "/Desktop/project/lib/python2.7/site-packages/django/db/models/fields/files.py", line 97, in save self.instance.save() File "/Desktop/project/src/accounts/models.py", line 80, in … -
Django 1.10 AdminEmailHandler infinite loop when DEBUG=False
I am facing a very strange behavior with Django 1.10 when setting DEBUG=False. Django logging utility is stuck in an infinite loop and is sending me tons of emails instead of only sending me one email. Actually it doesn't stop until I am stopping Apache. I tried downgrading to Django 1.9 and it solves the issue. I also tried explicitly setting an AdminEmailHandler in settings.py in order to enable email logging when DEBUG=True and it solves the issue. I am using Apache + mod_wsgi. Here is my (pretty basic) settings.py: import os BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) SECRET_KEY = "0m(!5fot^1kw)gs=rw%0)assk+y=s$)8y!g)l2b2i2@&#%kn25" DEBUG = False ALLOWED_HOSTS = ['127.0.0.1','172.18.0.2'] INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles',] MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware',] ROOT_URLCONF = 'hydra.urls' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, }, ] WSGI_APPLICATION = 'hydra.wsgi.application' DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'hydra_db', 'USER': 'juju', 'PASSWORD': 'pass', 'HOST': '172.18.0.1' }, } AUTH_PASSWORD_VALIDATORS = [ { 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', }, { 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', }, { 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', }, { 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', }, ] LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True … -
Django: empty formset in template
I'm building an application with Django and when using InlineFormSet with ClassBasedViews my route_formset is empty in template, but form is populated with data as it should be. What am I doing wrong? View class ReservationUpdateView(UpdateView): model = Reservation form_class = ReservationForm success_url = '/' def get_form_kwargs(self): kwargs = super(ReservationUpdateView, self).get_form_kwargs() kwargs.update({'request': self.request}) return kwargs def get(self, request, *args, **kwargs): """ Handles GET requests and instantiates blank versions of the form and its inline formsets. """ self.object = self.get_object() form_class = self.get_form_class() form = self.get_form(form_class) route_formset = RouteFormSet() return self.render_to_response( self.get_context_data(form=form, route_formset=route_formset)) def post(self, request, *args, **kwargs): """ Handles POST requests, instantiating a form instance and its inline formsets with the passed POST variables and then checking them for validity. """ self.object = self.get_object() form_class = self.get_form_class() form = self.get_form(form_class) route_formset = RouteFormSet(self.request.POST) if (form.is_valid() and route_formset.is_valid()): return self.form_valid(form, route_formset) else: return self.form_invalid(form, route_formset) def form_valid(self, form, route_formset): """ Called if all forms are valid. Creates a Recipe instance along with associated Ingredients and Instructions and then redirects to a success page. """ self.object = form.save() route_formset.instance = self.object route_formset.save() return HttpResponseRedirect(self.get_success_url()) def form_invalid(self, form, route_formset): """ Called if a form is invalid. Re-renders the context data with the data-filled … -
Django FBVs to CBVs
I'm refactoring my code and change all FBVs to CBVs. But one of them isn't look pretty well. So I'm asking for right CBVs for my example. Here my FBVs @login_required def profile(request, username): user_main = get_object_or_404(User, username=username) user_profile = UserProfile.objects.get_or_create(user=user_main)[0] form = UserProfileForm({'website': user_profile.website, 'picture': user_profile.picture}) if request.method == 'POST' and request.user == user_main: form = UserProfileForm(request.POST, request.FILES, instance=user_profile) if form.is_valid(): form.save(commit=True) return redirect(profile, username) else: print(form.errors) context_dict = {'user_main': user_main, 'user_profile': user_profile, 'form': form} return render(request, 'rango/profile.html', context=context_dict) Here's what I'm got: @method_decorator(login_required, name='dispatch') class ProfileView(DetailView, FormMixin): template_name = 'rango/profile.html' context_object_name = 'user_main' form_class = UserProfileForm def get_object(self, queryset=None): user_main = get_object_or_404(User, username=self.kwargs['username']) return user_main def get_context_data(self, **kwargs): context = super(ProfileView, self).get_context_data(**kwargs) context['user_profile'] = self.get_user_profile() return context def get_initial(self): user_profile = self.get_user_profile() return {'website': user_profile.website, 'picture': user_profile.picture} def get_form_kwargs(self): kwargs = super(ProfileView, self).get_form_kwargs() kwargs['instance'] = self.get_user_profile() return kwargs def post(self, request, *args, **kwargs): self.object = self.get_object() form = self.get_form() if form.is_valid and request.user == self.object: return self.form_valid(form) else: return self.form_invalid(form) def form_valid(self, form): profile = form.save(commit=True) return redirect('profile', self.kwargs['username']) def form_invalid(self, form): print(form.errors) return super(ProfileView, self).form_invalid(form) def get_user_profile(self): return UserProfile.objects.get_or_create(user=self.object)[0] I guess my way to solve this task looks horrible. Please show me the right way to make … -
Django pre_save not saving changes
I need to modify some value before saving model: @receiver( pre_save ) def model_pre_save( sender, instance, *args, **kwargs ): print instance.path # prints "old_path" instance.path = some_path() print instance.path # prints "new_path" But after save instance.path still equals "old_path". -
Invalid template
I am trying to send sms ,i am getting like this {"errors":[{"code":80,"message":"Invalid template"}],"status":"failure"} my code is: def sending_sms(number,message): url = 'http://api.textlocal.in/send/' msg = message post_fields=({"username":"*********","password":"******","numbers":number,"message":msg}) request = Request(url, urlencode(post_fields).encode()) print request json = urlopen(request).read().decode() print json return json -
Docker compose install requirements in a shared directory
I am having several containers, and each of my containers are having their own Dockerfile. Everytime I am building, using docker-compose build, each container runs its own requirements; either from a requirements.txt file (RUN pip install -r requirements.txt), or directly from the Dockerfile (RUN pip install Django, celery, ...). Many of the requirements are common in some of the containers (almost all). It is working perfectly, but there is a problem with build time. It takes almost 45 minutes to build every container from scratch. (lets say after I deleted all images and containers) Is there a way, to install all the requirements in a common directory for all containers, so that we dont install the common requirements each time a new container is building? Docker-compose I am using is version 2. -
Count # asserts executed when running py.test
When I run py.test on file like test_a(): ... test_b(): ... Py.test will say ... === 2 passed in 3.1415 seconds === .... I would like to know how many asserts have been tested, as one test_() function might have had multiple generated (for example in a for cycle, when testing new, faster implementation against slow, reference implementation of unit tested function. -
CSS: Django button and search bar inline
I have this html in Django template: <div id = "search_form"> <form action="" method="get" id = "search-form"> {% csrf_token %} {{ form.as_p }} {{ form.media }} </form> <button class="btn btn-primary" id="search_submit" name = "search_submit" type="submit" value = "submit" ><span class="glyphicon glyphicon-search"></span></button> </div> How to make search bar and button inline? This is not working: #search_submit { display: inline; } -
How to do user register validation in base.html django
With out calling register.html page. Placed that code in base.html. How to do validation in base.html using django? How can i acheive this. Please help me. If i call register.html page validation is working fine. i want to do aceive in base.html. base.html <form method="post" action="{% url 'register' %}"> {% csrf_token %} <table border="0"> <label for="name">Name:</label> <input type="text" id="username" name="username" value=""> {{ form.errors.name }} <label for="mail">Email:</label> <input type="text" id="email" name="email"> {{ form.errors.email }} <label for="password1">Password:</label> <input type="password" id="password1" name="password1"> {{ form.errors.password1 }} <label for="password2">Password:</label> <input type="password" id="password2" name="password2"> {{ form.errors.password1 }} </table> <button type="submit" value="Register">Register</button> <button type="button" onclick="window.location.href='/' ">Login</button> </form> forms.py class RegistrationForm(forms.Form): username = forms.RegexField(regex=r'^\w+$', widget=forms.TextInput(attrs=dict(required=True, max_length=30)), label=_("Username"), error_messages={ 'invalid': _("This value must contain only letters, numbers and underscores.") }) email = forms.EmailField(widget=forms.TextInput(attrs=dict(required=True, max_length=30)), label=_("Email address")) password1 = forms.CharField(widget=forms.PasswordInput(attrs=dict(required=True, max_length=30, render_value=False)), label=_("Password")) password2 = forms.CharField(widget=forms.PasswordInput(attrs=dict(required=True, max_length=30, render_value=False)), label=_("Password (again)")) def clean_username(self): try: user = User.objects.get(username__iexact=self.cleaned_data['username']) except User.DoesNotExist: return self.cleaned_data['username'] raise forms.ValidationError(_("The username already exists. Please try another one.")) def clean(self): if 'password1' in self.cleaned_data and 'password2' in self.cleaned_data: if self.cleaned_data['password1'] != self.cleaned_data['password2']: raise forms.ValidationError(_("The two password fields did not match.")) return self.cleaned_data -
Scrapy blocks script execution after scraping (ReactorNotRestartable Error)
Identical question here. (But it's for old version!) I'm executing scrapy spider from celery task in Django application. I'm giving URL to spider as domain=url while calling it and dump the result into DB using DjangoItem. Everything works perfectly if I call the scrapy spider at the first time. While calling it the second time I'm getting this error: Traceback (most recent call last): File "/home/msn/Documents/Works/project/site/content/tasks.py", line 194, in emails_spider_crawl process.start() # this will stop twisted reactor after crawling File "/home/msn/Documents/Works/project/venv/lib/python3.5/site-packages/scrapy/crawler.py", line 280, in start reactor.run(installSignalHandlers=False) # blocking call File "/home/msn/Documents/Works/project/venv/lib/python3.5/site-packages/twisted/internet/base.py", line 1198, in run self.startRunning(installSignalHandlers=installSignalHandlers) File "/home/msn/Documents/Works/project/venv/lib/python3.5/site-packages/twisted/internet/base.py", line 1178, in startRunning ReactorBase.startRunning(self) File "/home/msn/Documents/Works/project/venv/lib/python3.5/site-packages/twisted/internet/base.py", line 687, in startRunning raise error.ReactorNotRestartable() twisted.internet.error.ReactorNotRestartable What I've tried so far: tasks.py settings = get_project_settings() settings.overrides['ITEM_PIPELINES'] = { 'web_scraper.web_scraper.pipelines.EmailSpiderPipeline': 300, } process = CrawlerProcess(settings) process.crawl(WebScraperSpider, domain=url) process.start() items.py class WebScraperItem(DjangoItem): django_model = Info pipelines.py class EmailSpiderPipeline(object): def process_item(self, item, spider): p = WebScraperItem() p['email_address'] = list(item['email_address'])[0] p.save() # return p I found many identical questions but didn't find any solution regarding my use case! Can someone shed some light on how this problem can be solved? Or Am I missing something? [Note: I'm using scrapy 1.3.0] Thanks a bunch! Filetree: /mysite(Django Project) /mysite __init__.py …
