« first day (860 days earlier)      last day (1328 days later) » 

12:17 AM
0
Q: What is numpy's equality operator doing here?

ClaudiuConsider the following: >>> (np.arange(10) == [[0], [5], [4]]).astype(float) array([[ 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.], [ 0., 0., 0., 0., 1., 0., 0., 0., 0., 0.]]) I understand the effect - the result is...

0
Q: pandas: group by custom categories, then pivot?

RichardI have a dataframe that looks like this: Company Name Category Amount 0 Teva Limited Consultancy 650.000 1 Teva Limited Consultancy 5.300 2 Eli & Company Ltd Events ...

0
Q: Why can't I import pandas in Winpython IDLE?

Cofeinnie BondaI downloaded Winpython and installed. Then I opened IDLEX (Python GUI) and want to import pandas, but it shows: Traceback (most recent call last): File "", line 1, in import pandas ImportError: No module named pandas Why is this? Thanks.

 
12:37 AM
1
Q: Python reshaping a list of strings based on delimiter

CalamariI have pipe delimited data being returned from a Rest API call: data = ['Bob|37', 'Eric|40'] I extracted the header info from the first "line" and split it into a list: column_list = ['Name','Age'] I am trying to read this into a pandas dataframe: pd.DataFrame(data,columns=column_list) U...

 
1:03 AM
0
Q: Do I lose the copyleft protections of GPLv3 when I publish my code on PyPI?

dln385I created a Python package and licensed it under GPLv3. When I created an account on PyPI, I had to agree to the following: The PSF is free to use or disseminate any content that I upload on an unrestricted basis for any purpose. In particular, the PSF and all other users of the web site ...

 
1:20 AM
1
Q: When is copy on write invoked for python multiprocessing across class methods

alexrsonI have a bit of multiprocessing Python code that looks a bit like this: import time from multiprocessing import Pool import numpy as np class MyClass(object): def __init__(self): self.myAttribute = np.zeros(100000000) # basically a big memory struct def my_multithreaded_analysi...

 
 
1 hour later…
2:22 AM
0
Q: Elementwise Sampling with map_fn Slow

TFUserSay that I want to sample a matrix with each entry sampled from a distribution defined by an entry in another matrix. I unroll my matrix and apply map_fn to each element. With a relatively small matrix (128 x 128), the following gives me several PoolAllocator warnings (GTX TITAN Black) and does n...

 
2:37 AM
0
Q: Tweaking the seaborn graph

liv2hakI am trying to plot the timeseries graph of internet traffic from 11 cities in Europe. I got access to the data set from internet traffic data of 11 european cities # !/usr/bin/env python3.4 # -*- coding: utf-8 -*- import pandas as pd import seaborn as sns import matplotlib.pyplot as plt df =...

 
2:55 AM
0
Q: embedding Matplotlib in PyQt5 using anaconda3

icantcodeI'm trying to make a simple program with matplotlib embedded in a PyQt5 gui. I am using pycharm with anaconda as the interpreter. I installed PyQt5 using the binary installers here. My imports are import sys from PyQt5 import QtCore, QtGui, QtWidgets, uic import numpy as np from matplotlib impor...

 
3:18 AM
0
Q: numpy: broadcasting into multiple inner products and inverses

DathosPachyI have arrays e, (shape q by l) f (shape n by l), and w (shape n by l), and I want to create an array M where M[s,i,j] = np.sum(w[s] * e[i] * e[j]), and an array F, where F[s,j] = np.sum(w[s] * f[s] * e[j]). Both are easy enough to do by, for instance, looping through elements of M, but I want t...

0
Q: what is the difference between ['[a,a,a]','[b,b,b]'] and [[a,a,a],[b,b,b]] in python?

HimanUCCIn my program, when I debug my code, it seems somewhere in my code I generate X1=['[a,a,a]','[b,b,b]'] and somewhere else I generate X2=[[a,a,a],[b,b,b]] and when I want to add this two lists and then use numpy.array(X1+X2) I get error. The problem is I don't know what is wrong with my code as I ...

0
Q: Shift irregular timeseries forward one (custom) business day in Pandas

Doctor JI have a DataFrame with an irregular timeseries index of dates: import numpy as np import pandas as pd from pandas.tseries import offsets, holiday df = pd.DataFrame(index=pd.to_datetime(['20160629', '20160630', '20160701'])) Empty DataFrame Columns: [] Index: [2016-06-29 00:00:00, 2016-06-30 0...

 
3:50 AM
0
Q: IndexError when calling fft2 in numpy

cruiser0631I've been having IndexError when trying to do a discrete Fourier transform to some images with numpy.fft.fft2. It works for some images, but the error is raised for most of the rest. Here are my codes: import matplotlib.pyplot as plt import numpy as np import cv2 fig = cv2.imread('test_img4.tif...

 
4:17 AM
1
Q: Inserting element into arbitrary position of an n-dimensional array

Mathews24I am using Python 2.7 to create a complex-valued (m x n)-dimensional array without an initially known fixed size (i.e. m and n are not known ahead of time) which will have particular elements assigned different values. Thus, I will be changing existing elements or adding new elements to this arra...

 
4:34 AM
0
Q: How to visualize an distance matrix in Python?

Mark LiI have an distance matrix (as DataFrame) as below for which I would like to create an plot like the second graph to visualize the distances. Is there any available plot object in Python can visualize distance matrix?

 
4:55 AM
0
Q: Removing wireframe without gaps in matplotlib

mooI want to create a smooth cylinder using matplotlib/pyplot. I've adapted a tutorial online and produced the following minimal example: from numpy import meshgrid,linspace,pi,sin,cos,shape from matplotlib import pyplot import matplotlib.tri as mtri from mpl_toolkits.mplot3d import Axes3D u,v = ...

 
5:48 AM
0
Q: filter DataFrame using dictionary

DemauntI am new to pandas and python. I want to use dictionary to filter DataFrame import pandas as pd from pandas import DataFrame df = DataFrame({'A': [1, 2, 3, 3, 3, 3], 'B': ['a', 'b', 'f', 'c', 'e', 'c'], 'D':[0,0,0,0,0,0]}) my_filter = {'A':[3], 'B':['c']} When I call df[df.isin(my_filter)] ...

 
6:00 AM
0
Q: Python: Optimizing simple image scraping code

Lachy VassI'm fairly new to python. I wrote this code over the last few days. I learned a lot, and it works how I expect, however I suspect it's woefully inefficient: import requests, bs4, os os.chdir('../../Desktop') wallpaperFolder = os.makedirs('Wallpapers', exist_ok=True) pageCount = 1 GalleryUrl = ...

0
Q: Jupyter does not import libraries after installing Python 3

gurlukI was using SciPy, NumPy, Matplotlib, SymPy, Numba on Python 2. I have installed Python 3, however Jupyter could not import these packages. For example, I am getting: ImportError: No module named 'sympy' I have installed and activated packages over again with the commands: conda create -n foo ...

 
6:18 AM
0
Q: Entity Reference Character in HTML

NishantInput supplied as Text # Python 2.7 >>> bs4.BeautifulSoup("H&W Insurance") <html><body><p>H&amp;W Insurance</p></body></html> # Python 3.5.2 >>> import lxml.html >>> h = lxml.html.fromstring("H&W Insurance") >>> lxml.html.tostring(h) b'<p>H&amp;W Insurance</p>' BeautifulSoup and lxml has esca...

 
6:36 AM
0
Q: Trying to plot a defined function

Osvaldo PanicciaI have a problem. I try to plot a function for different values of d. I have defined d as: d = np.arange(0.0, 100.0, 0.01) But I still get the same error: TypeError: only length-1 arrays can be converted to Python scalars That is my script: import pylab import numpy as np import scipy ...

0
Q: Display two Bar Chart one by one in MATPLOTLIB

Vigneshwaran ThenrajUsing Pivot Table, I am plotting Barchart. I have two Independent Pivot Table and need to produce two barchart one by one as PNG Image, The Chart is generated but it wont display as expected WHAT I TRIED IS: Snippet as Follow: # Ploting Activity Begins fig = plt.figure() ax1 = fig.add_s...

0
Q: Accessing objects and tuples from a function

keven renI have a feeling there should be a very easy way to access specific elements from an evaluated function. A very simple example of what I am trying to achieve is def func(x): a = 2*x b = x*x return 1, 10, 100, (a,b) I define a value for x and the function returns a set of values and...

 
6:57 AM
-3
Q: Python Group By Consecutive Count

Vivek KalyanaranganHere's the data <table border="1"> <tr><td>ID</td><td>Type</td> </tr> <tr><td>1</td><td>In</td> </tr> <tr><td>1</td><td>In</td> </tr> <tr><td>1</td><td>Out</td> </tr> <tr><td>1</td><td>In</td> </tr> <tr><td>2</td><td>Out</td> </tr> <tr><td>2<...

 
7:12 AM
0
Q: pandas module's backward compatibility on AWS

sidharth228I used pandas version 0.18.1 to write a python (version 2.7) code on my machine. I want to run it on AWS EC2, but the code throws an exception in read_excel() method of pandas module regarding the number of parameters inspite of working perfectly on my machine. I suspect that pandas version i...

0
Q: SI1145 Adafruit UV Sensor Library porting from Ardunio to Python SMBUS

Davies-BarnardFirst post here. I originally asked this question on the Raspberry Pi Exchange because of its specific nature but it was suggested it would be better seen here so here goes. I have spent the day trying to port the Adafruit UV Sensor Library (Link) from Arduino to Python and SMBUS for the Raspbe...

0
Q: Python: What is the difference between using `[data2]` and `[[data2]]` with `groupby` on a Pandas DataFrame?

Mark BlatnikI am working through a Python for data analysis tutorial and want some clarification on the output I get from using [data2] and [[data2]] when using groupby. A link to the image of my jupyter notebook Thank you for your help.

0
Q: Split dataframe in 3 subgroups

DemauntI want to split raw dataframe into 3 subgroups: train, test, validate I see three solutions, but afraid they are not correct and may cause bottle neck effect 1) add dictionary with keys my_dict = {'train':raw_df.loc[start:end], 'test':raw_df.loc[start:end], 'val':raw_df.loc[start:e...

0
Q: Can not import matplotlib with pyenv ipython notebook

channa lyI am a ruby developer I start learning python for a project. I install pyenv, pyenv virutualenv, ipython, notebook, matplotlib. Everything worked fine except the part I called import matplotlib.pyplot as plot Then I 've got this error from notebook: RuntimeError ...

0
Q: How to write gui in Python for avrdude (usbtiny programmer)?

mayanksagari want to program Atmega 328 using usbtiny programmer . i want to know how to kind of how to go about creating a gui in Python using pyqt4 to program some specific varibles in the arduino program from the gui and program it to the Atmega 328 .

 
7:57 AM
1
Q: float values error after numpy vstack

Peter I have a problem while using numpy vstack for float values with exponent e-5. Both item_array and date_array are numpy.ndarray. When I use data = np.vstack((date_array,item_array)), the element 3.04871703e-05 become to 3.0487170344. Other elements are good. Does anyone can help me to fix this is...

0
Q: Does Pandas.Series.unstack() affect data types?

CrolleI'm working on a MultiIndex Series containing mixed-types values (timedeltas and int): char 7 a 103 minutes s 63 9 a 129 minutes s 211 10 a 106 minutes s 63 Name: timestamp, dtype: object Index : MultiIndex(levels=[...

0
Q: How are signals and KeyboardInterrupt handled in Python?

dontlooI have two Python scripts foo.py and bar.py, foo.py will call bar.py via os.system(). #foo.py import os print os.getpid() os.system("python dir/bar.py") #bar.py import time time.sleep(10) print "over" Say the pid of foo.py is 123, if the program terminate normally, it'll print 123 over ...

 
8:22 AM
0
Q: Python. Guess the Number

Ilia_MochalovI have been using Python 3 to learn programming, this is very basic task, but I want to gain good programming patterns/habits at the very beginning. The program will first randomly generate a number unknown to the user. The user needs to guess what that number is. Program interacts with the user...

0
Q: Using 'apply' with arrays inside Pandas dataframe elements

MichaelI am working with pandas dataframes which contain arrays inside the dataframe elements. I'm trying to "apply" a function to these elements, and then return an array. But I have some very inconsistent behavior. The function runs okay the first few times, but then it fails. Here is my code: im...

 
8:53 AM
1
Q: Convert output of count method into a percentage value

adeleIf I run .count() on an object: animals = all_vals[['Dog','Cat','Mouse','Snake']].count() and receive this output: animals Out[49]: Dog 173 Cat 13 Mouse 42 Snake 359 dtype: int64 How do I then get the percentage occurrence of each animal? Can I send the output of .count into...

0
Q: Linear and circular convolution in Python

Medulla OblongataI'm trying to perform linear convolutions in Python by comparing the results from FFTs and convolution functions. Python's scipy.signal.fftconvolve automatically does the necessary zero padding. If we do the calculation using only FFTs, we add a length of zeros after our input signal. Is there a...

 
9:09 AM
0
Q: Python comparing a string or list of strings to a string or list

vivasI'm using a selfmade python program for learning new words in new languages. Originally I made this program for a university course but now I'm just trying to imrove it and make the code "prettier" and perhaps some small functionality improving. :) The basic principle is that the program reads f...

 
9:21 AM
0
Q: Gets stats of the data with UNIX timestamp

Sitz BlogzI have this huge data set which has dates for over 100 days and several timestamps. The datetime format is in UNIX format. The datasets are logs of some login. I am trying to get some stats like: total log counts per hour & unique login ids per hour. log count with choice of hours i.e. 24hrs...

 
9:38 AM
0
Q: pandas group by: usage in this instance

Asher11despite the extensive help provided here (http://pandas.pydata.org/pandas-docs/stable/groupby.html) and here (http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html) I was unable to figure out how to do the following: given this dataset (df): import pandas as pd im...

0
Q: Use Apply on a SeriesGroupBy Object where conditions are met

Michael_OI have a DataFrame df1: df1.head() = id ret eff 1469 2300 -0.010879 4480.0 328 2300 -0.000692 -4074.0 1376 2300 -0.009551 4350.0 2110 2300 -0.014013 5335.0 849 2300 -0.286490 -9460.0 I would like to create a new column that contains the nor...

 
10:03 AM
-1
Q: Unhashable type: list on using Counters in python

Karl MaxI have a list which is something like this: list = [ ['abc'] , ['def'] , ['ghi'] , ['abc']...] I want to find the frequency of each element in the list. So the output should be something like: { ['abc']:2, ['def']:1, ['ghi']:1,...} the key value pairs may be interchanged that's not a proble...

0
Q: putting .size() into new column python pandas

hsquaredI am very new to python (and to stack overflow!) so hopefully this makes sense! I have a dataframe which contains years and names (amongst otherthings however this is all I am interested in working with). I have done df = df.groupby(['year', 'name']).size() to get the amount of times each names...

 
10:22 AM
1
Q: Edit tensorflow inceptionV3 retraining-example.py for multiple classificiations

WboyHello kind people :) I've spent a few days searching many stackoverflow posts and the documentation, but I could not find an answer to this question. Would greatly appreciate any help on this! I have retrained a tensorflow inceptionV3 model on new pictures, and it is able to work on new images b...

 
10:39 AM
0
Q: how to change the whole dataframe values based on another one in Python

SaplingI have two dataframes: df = pd.DataFrame({'A': [1,2,3,5,4,2], 'B':[1,2,4,1,2,3], 'C':[2,3,1,4,2,3]}) df1 = pd.DataFrame({'num':[1,2,3,4,5],'col':['red','red','blue','orange','orange']}) They look like this: df: A B C 0 1 1 2 1 2 2 3 2 3 4 1 3 5 ...

 
10:50 AM
0
Q: Pandas: I am trying to open a specific .txt file stored in a zip file on an ftp site

Michael PerdueI need to concatenate---into a single frame---every produkt_monat_Monatswerte_18910101_20110331_00003.txt file in each of the zip files from this ftp site. This is the code that I am using so far: import pandas as pd from pandas.io.parsers import * import glob import requests from zipfile im...

0
Q: Odd behaviour of pcolormesh with coordinates

Till HoffmannI would like to plot a color mesh using matplotlib's pcolormesh function. However, it gives different results depending on whether I specify coordinates or not. import numpy as np from matplotlib import pyplot as plt np.random.seed(9) x = np.arange(10) z = np.random.normal(0, 1, (10, 10)) plt....

 
11:10 AM
0
Q: Groupby in python pandas: Fast Way

NáthaliI want to improve the time of a groupby in python pandas. I have this code: df["Nbcontrats"] = df.groupby(['Client', 'Month'][Contrat].transform(len) The objectif is to Count how many contrats a client has in a month and add this information in a new column (Nbcontrats) Client: client code M...

0
Q: importing whole python module doesn't allow using submodules

user3524489My question is specific to scikit-learn python module, but I had similar issues with matplotlib as well. When I want to use sklearn, if I just do 'import sklearn' and then call whatever submodule I need, like ' sklearn.preprocessing.scale()', I get an error "AttributeError: 'module' object has ...

0
Q: ordering in boxplot according to an object

progsterI would like to have some grouped boxplot in a pandas df. var2 is an object, and I would like to display the boxplot in alphabetical order according to var2 order import seaborn as sns sns.set_style("whitegrid") ax = sns.boxplot(x="var1", y="var2",order=???, data=df) without putting manual...

0
Q: pandas: conditions with str.contains()

ldevyataykinaI have data url https://s.youtube.com/api/stats/qoe?fexp=9416891%2C9419451%2C9422596%2C9428269%2C9428398%2C9428492%2C9431012%2C9431657%2C9431674%2C9433096%2C9433380%2C9433946%2C9434803%2C9435467%2C9435526%2C9435876%2C9436275%2C9436302%2C9436484%2C9437553%2C9437967%2C9438327%2C9438699%2C9439362%...

 
11:44 AM
0
Q: How to perform stencil computations element-wise on a matrix in Theano?

AnonynagsI have the following blurring kernel I need to apply to every pixel in an RGB image [ 0.0625 0.025 0.375 0.025 0.0625 ] So, the pseudo-code looks something like this in Numpy for i in range(rows): for j in range(cols): for k in range(3): final[i][j][k] = image[i-2][...

 
11:56 AM
0
Q: Does python cache repeatedly accessed files?

drachenbachI was wondering if python is smart enough enough to cache repeatedly accessed files. E.g. when reading the same CSV with pandas or unpickling the same file multiple times. Is this even in python's responsibility or should the operating system take care of it? Thank you!

 
12:11 PM
0
Q: Set Values for datetime Index Slices

joshI have a datetime indexed pandas DataFrame and I wish to replace different datetime ranges of values depending on the column. My DataFrame that stores the values looks as follows, df1 Out[1]: IL15 IL09 IL06 2016-06-23 00:00:00 1 1 1 2016-06-23 00:01:00 1 1 ...

 
12:28 PM
0
Q: pandas dataframe to mariadb database with to_sql and sqlalchemy

nlucknI want to use the pandas function to_sql to write a dataframe into a MariaDB database. My Python code inside PyCharm looks as follows: import pandas as pd import mysql.connector from sqlalchemy import create_engine myd = pd.read_csv('/[path]/[filename].csv') engine = create_engine('mysql+mysql...

0
Q: right join with pandas

Night Walker stores = [[232, '2016-02-05 04:30:00', 'Test User', 1], [332, '2016-02-06 04:30:00', 'Test User', 2], [432, '2016-02-07 04:30:00', 'Test User', 3], [532, '2016-02-08 04:30:00', 'Test User', 4], [632, '2016-02-...

0
Q: controling the x ticks date values

M.MI have the following code: ax.set_xticklabels(getLabels(s,t),rotation=20) where getLabels(s,t) is defined as: def getLabels(s,t): #s and t are unix time stamps labels =[] for x in pd.date_range(start=s, end=t, freq='3M'): labels.append(str(x).replace(" 00:00:00","")) print...

0
Q: Python - What am I doing wrong?

tom790def ICP(x): numofrepeat=0 warning=0 x_str = str(x) while x_str[numofrepeat]=='2' or x_str[numofrepeat]=='3' or x_str[numofrepeat]=='5' or x_str[numofrepeat]=='7': if (x[numofrepeat]==0): warning=warning+1 if warning>1: numofrepeat=len(x)+1 if warning>1: return("false")...

0
Q: Filtering similar images in 2 numpy.ndarray in effective way

Maxim HaytovichCurrently I play with some ML algorithms and I have 2 sets of images (train and validation sets) which I store in in np.ndarray. Images are greyscale, size is 28x28. I have about 200000 of train images and smth like 20000 of validation images. I know that my dataset is not ideal and: - In each da...

0
Q: How to concat Pandas dataframe columns

ArchieTigerHow to concat columns of different data types in a Pandas dataframe such that if column number is concatenated with column operator, and I do a groupby('user').sum(), I can have the appropriate aggregation: number operator user id 1 193 - A 2 332 ...

0
Q: Count time to some event using pandas

ldevyataykinaI have data date url 2016-07-01 11:43:09 https://www.youtube.com/api/stats/ads?ver=2&ns=1&event=2&device=5&content_v=cmqQy3v8KvM&el=detailpage&ei=bld2V--RJoWHW4OQlbAN&devicever=11.23&format=2_2_1&break_type=1&conn=3&cpn=ifKDc6HI6TsnATCH&lact=8504&m_pos=0&mt=0&p_h=360&p_w=640&rwt=1467373422397%2C...

0
Q: Matplotlib plot bar graph with maximum length

prathmesh.kallurkarI am using Matplotlib to plot a bar graph. Most of my data values are in the range of -10 to +30. However, I have two data values that are around -300. When I plot my data, the -300 data value bar looks too big, and it hides the insights of the other bars. Is there a way that I can plot all bars...

0
Q: appending values to a new dataframe and making one of the datatypes the index

StaceyI have a new Data-frame df. Which was created using: df= pd.DataFrame() I have a date value called 'day' which is in format dd-mm-yyyy and a cost value called 'cost'. Could someone let me know how I can append the date and cost values to the df and assign the date as the index please? So fo...

 
12:56 PM
0
Q: How to plot age distribution with pandas

dimazubrikI have Data Frame, which contains 2 columns: age and gender. How can I plot age distribution for each gender?

 
1:11 PM
stackoverflow.com/questions/38145142/… OP is asking basic questions, for which answers can be found by checking on google
 
1:30 PM
2
Q: Project Euler 1: A sledgehammer to crack a nut

N3buchadnezzarThe problem Project Euler 1 is one of the most asked questions on site. However I wanted to solve the more general problem of division. Multiples of a list If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23....

0
Q: Arbitrary colorbar norm

jadsqI have data that is in the range -70,0 that I display using imshow() and would like to use a non-linear colorbar to represent the data as I have paterns both in the -70,-60 range and -70,0 range. I would like the easiest way to rescale/renormalize using an arbitrary function (see example) the col...

0
Q: consolidate multiple pandas dataframes with unpredictable column headers/order into single frame or list

joshi123I have searched a lot on this topic and been unable to find any existing question which addresses this specific problem. Nb I am very new to coding and python. I'm reading multiple excel source data files which are similar in structure, but the column headers are unpredictable in that the orde...

 
1:47 PM
1
Q: Inverse filtering using Python

khpeekGiven an impulse response h and output y (both one-dimensional arrays), I'm trying to find a way to compute the inverse filter x such that h * x = y, where * denotes the convolution product. For example, suppose that the impulse response h is [1,0.5], and the output is a step function (that is, ...

0
Q: Action with pandas warning

ldevyataykinaI try to delete some column and convert some value in column with df2.drop(df2.columns[[0, 1, 3]], axis=1, inplace=True) df2['date'] = df2['date'].map(lambda x: str(x)[1:]) df2['date'] = df2['date'].str.replace(':', ' ', 1) df2['date'] = pd.to_datetime(df2['date']) and to all this string I get...

 
2:04 PM
0
Q: How to remove square bracket from pandas dataframe

KrugerBrI came up with values in square bracket(more like a list) after applying str.findall() to column of a pandas dataframe. How can I remove the square bracket ? print df id value 1 [63] 2 [65] 3 [64] 4 [53] 5 [13] 6 ...

 
2:26 PM
0
Q: Dask: low CPU usage and multiple threads?

NoobieI am using dask as in how to parallelize many (fuzzy) string comparisons using apply in Pandas? Basically the code looks like: dmaster = dd.from_pandas(master, npartitions=4) dmaster = dmaster.assign(my_value=dmaster.original.apply(lambda x: helper(x, slave), name='my_value')) dmaster.compute(g...

0
Q: modifying sparse matrix using advanced indexing in python

aaragonI'm trying to use advanced indexing to modify a big sparse matrix. Say you have the following code: import numpy as np import scipy.sparse as sp A = sp.lil_matrix((10, 10)) a = np.array([[1,2],[3,4]]) idx = [1,4] A[idx, idx] += a Why this code doesn't work? It gives me the error ValueError:...

0
Q: How to change a 'LinearSegmentedColormap' to a different distribution of color?

DmanI am trying to make a color map that 'favors' lower values, i.e. it takes longer to get out of the darker color to get to the light color. At the moment I am using this as the colormap: cmap = clr.LinearSegmentedColormap.from_list('custom blue', ['#ffff00','#002266'], N=256) I am plotting this...

0
Q: TypeError: 'Series' objects are mutable, thus they cannot be hashed

cjm2671Code: wd = {'ewmac16': 0.032178851993959409, 'ewmac64': 0.12865803853630536, 'ewmac8': 0.10429234477813946, 'Carry': 0.53455719563599335, 'ewmac2': 0.069251450564870309, 'ewmac4': 0.088705814463579821, 'ewmac32': 0.042356304027152451} instruments['corn'].account_curve(weights=pd.Series(wd)) Er...

 
2:56 PM
0
Q: coloring matplotlib scatterplot by third variable with log color bar

evtohI'm trying to make a scatterplot of two arrays/lists, one of which is the x coordinate and the other the y. I'm not having any trouble with that. However, I need to color-code these points based on their values at a specific point in time, based on data which I have in a 2d array. Also, this 2d a...

-1
Q: Live Plot Updating in Python

Michael HendersonI want to be able to capture the video source from my webcam and have a Plot updating live based on the live webcam data. Here is my current code, it does not work atm. Essentially I want to have a live plot of the chrominance values based on the params I feed it. Hopefully someone with more k...

0
Q: Setting an image at center of figure using matplotlib Figure.figimage

zeppelin_dI'm trying to set an image, a watermark essentially, at the center of a matplotlib Figure. I would like the center of the image to be at the center of the figure. The problem appears to be a difference between the figure size and dpi of 10" x 10" and 960 x 960 pixels and the saved image which has...

 
3:16 PM
0
Q: pandas: reindex panel with dataframe index

Steve LorimerI have a panel with the Items index being days. <class 'pandas.core.panel.Panel'> Dimensions: 1260 (items) x 6 (major_axis) x 6 (minor_axis) Items axis: 2011-06-27 00:00:00 to 2016-06-28 00:00:00 Major_axis axis: BP to XOM Minor_axis axis: BP to XOM Index: DatetimeIndex(['2011-06-27', '2011-0...

0
Q: How to remove the extra row (or column) after transpose() in Pandas

Helena KAfter using transpose on a dataframe there is always an extra row as a remainder from the initial dataframe's index for example: import pandas as pd >>> df = pd.DataFrame({'fruit':['apple','banana'],'number':[3,5]}) >>> df fruit number 0 apple 3 1 banana 5 >>> df.transpose()...

0
Q: Difference two dataframes based on two columns Python Panda

Proton MailDf1 A B C 1 1 'a' 2 3 'b' 3 4 'c' Df2 A B C 1 1 'k' 5 4 'e' Expected output (after difference and merge of Df1 and Df2) i.e. Df1-Df2 is output A B C 1 1 'a' 2 3 'b' 3 4 'c' 5 4 'e' The difference should be based on two columns A and B and...

0
Q: IPython FITS file plotting gives different results

Anik HalderI am facing a problem while running a script ( please find the code below ). I am trying to plot an array of values, write it into a FITS file format, read it back again and plot it ---> I don't get the same plots! If you could please help me with this it would be great. The following are the ...

0
Q: OpenCV - python 3.x and windows - what version of Numpy?

user1700890I downloaded openCV from http://www.lfd.uci.edu/~gohlke/pythonlibs/#opencv and installed it. When I tried to run it: import cv2 I got error message: RuntimeError: module compiled against API version a but this version of numpy is 9 Traceback (most recent call last): File "<input>", line 1, in <...

0
Q: Successfully installed numpy and pandas, but cannot call pandas in Python Shell

user6538830I am using Python 2.7.12 on a Windows 7 machine I need to run a .py script that calls arcpy, numpy and pandas. I have not been able to import arcpy using Anaconda, so I deleted Anaconda from my machine and am trying to import pandas and numpy using the python shell. I figured out how to use ...

0
Q: Delete column(s) from very large CSV file using pandas or blaze

Alexander WhatleyI have a very large csv file (5 GB), so I do not want to load the whole thing into memory, and I want to delete one or more of its columns. I tried using the following code in blaze, but all it did was append the resulting columns to the existing csv file: from blaze import Data, odo d = Data("m...

0
Q: Manual set up colormap for color-bar Matplotlib

madf19I would like to create the following colormaps. mycolorbar1 = [dark-violet light-green dark-red dark-blue] mycolorbar2 = [dark-brown dark-red] then I can use them in the following part of my code. plt.tricontourf(triang1, Z1,v1,cmap=mycolorbar1, norm=plt.Normalize(vmin=v1mi,vmax=v1ma)) pl...

1
Q: Using ols function with parameters that contain numbers/spaces

ThomasI am having a lot of difficulty using the statsmodels.formula.api function ols(formula,data).fit().rsquared_adj due to the nature of the names of my predictors. The predictors have numbers and spaces etc in them which it clearly doesn't like. I understand that I need to use something...

 
4:14 PM
0
Q: 3D plot aspect ratio [matplotlib]

Hayk HakobyanI am using the code written here to make a plot like this one below. The problem is, that I want to adjust an aspect ratio, namely, to stretch it along the z-axis, so that all the stacked images are more or less visible. Is there an easy way of doing that?

0
Q: Using for loop in a data frame

NilI have two separate data frames one of which consists of year starting from 1900 to 2012 and in another data frame i am having 8 different flow measuring gauging stations with flow data. I need to extract the annual maximum flow data for each gauging stations and then populate them into a single...

0
Q: text interpreted as boolean (pandas, python 3)

Sjaak DalensI am using pandas with read_csv. It interprets strings as boolean if all values are either "true" or "false". How can I prevent this? My data.csv file content: String1,String2 true,false true,false true,true Code import pandas df = pandas.read_csv("data.csv", true_values=None, false_values=N...

 
4:41 PM
0
Q: Set value of a row in a Pandas dataframe equal to that of a row in a different dataframe

GreenGodotI have two dataframes, both have the same amount of columns and contain text data. The problem is that the data in the second dataframe is missing details: A B 1 Bob Hoskins 2 Laura Hogan 3 Tom Jones A B 1 Bob x 2 Bob x 3 Bob x 4 Laura x 5 ...

0
Q: Discrete sliders for multiple sliders, python

SuperThe problem of creating a single discrete slider has been solved here. Can I make matplotlib sliders more discrete? I would like to be able to create multiple discrete-sliders. In the python slider_demo pages, (http://matplotlib.org/examples/widgets/slider_demo.html), multiple sliders are ac...

 
5:03 PM
0
Q: Snakes on a prime

LegatoThe challenge is to find and print the largest palindrome prime under 1000. def reverse(integer): result = 0 while (integer != 0): result *= 10 result += (integer % 10) integer //= 10 return result def is_palindromic(integer): return integer == reverse(i...

 
5:26 PM
0
Q: Pandas Column/Index Values With Dollar Signs

Dance Party2Given the following data frame: import pandas as pd df=pd.DataFrame({'A':['$0-$20','$20+']}) df A 0 0−20 1 $20+ How can I get the first value (0-20) to display with the dollar signs, as I originally specified? It actually displays the zero in a strange font and I'm not sure why it work...

0
Q: Add shared row headers in matplotlib

CarolineI haven't been able to find a way to add a shared row header across multiple subplots, as in the linked figure. I don't want to use the xlabel of any plot, both because I need the label and because they don't align in the center of the plot. I'd appreciate any suggestions. Grid-Layout

1
Q: Encoding a list of frequencies as WAV audio

Solid CoderThis code gets a list of signals with predefined frequency from struc.num I need to encode it to wave format, so i use this code. This works, but i dont sure it is a correct decision to encode like this in python. Is there any ways to do it better? def compiler(): import out print("Loa...

0
Q: Dropping All Rows of Data after List number in python pandas

brandogIm am pretty sure I am going about this all wrong, but, this is what I have. Below shows the code I currently have, first I have a dataframe which has only one cell, the index row of interest (ie 75 or 80 or 32), then I turn it into a list, then I drop that row from my dataframe. Now I need to d...

0
Q: Pandas GroupBy date range depending on each row

dlealI would like to do the following: for a data frame that looks like this: df = pd.DataFrame({"ID":["A", "A", "C" ,"B", "B"], "date":["06/24/2014","06/25/2014","06/23/2014","04/07/1999","07/02/1999"], "value": ["3","5","1","7","8"] }) I would like to group by date all those observations that ar...

 
5:42 PM
0
Q: How do I print numpy array

Joe Half FaceI have this code in ipython notebook: print("hello") def load_letter(folder, min_num_images): """Load the data for a single letter label.""" image_files = os.listdir(folder) dataset = np.ndarray(shape=(len(image_files), image_size, image_size), dtype=np.float32) ...

 
trolol
 
0
Q: Find a python transformation function or numpy matrix to transform skewed normal distribution to normal distribution

embI have an input dataset (DataFrame / numpy matrix) that has a skewed normal distribution. I am trying to find the python transformation function (or numpy matrix) which will transform the input dataset to a normal distribution with no skew. I have looked at curve_fit (in scipy.optimize) and am ...

0
Q: How to sort groupby using pandas

aliceI have a code like this: grouped=L2014.groupby(['state','NAME']) grouped.mask.value_counts(normalize=True,).sort_index() results like this: state NAME mask CO Adams False 0.407195 True 0.592805 Alamosa False 1.000000 ...

 
6:17 PM
0
Q: Pandas to MatPlotLib with Dollar Signs

Dance Party2Given the following data frame: import pandas as pd df=pd.DataFrame({'A':['$0-$20','$20+']}) df A 0 0−20 1 $20+ I'd like to create a bar chart in MatPlotLib but I can't seem to get the dollar signs to show up correctly. Here's what I have: import matplotlib.pyplot as plt import nump...

 
6:40 PM
0
Q: Pandas - Line Graph Subplots with Multiple Indexes

DeadCakeI want to create line graph subplots with each Code per Customer having it's own subplot in a separate file for each customer. Date Code Customer Purchases 1 6/22/2016 6-BZt Piggy 8 2 6/22/2016 7rTPn Piggy 1 3 6/22/2016 Hb4vZ ...

0
Q: Converting File Number to Date--Python/matplotlib

P. ThompsonI'm currently working with files of data that are chronologically sorted by number, starting with 0000000120 and moving upwards in increments of 120 (so the next is 0000000240, etc). Each file moves forward 5 days, so if 0000000120 is 1/1/02, then 0000000240 would be 1/5/02 and so on. I'm trying ...

-1
Q: Show only unique values in column

Yared J.I want to filter out repeated values and list only unique values in my DataFrame. Lets say I have a df like this: A B 0 1 3 1 2 4 2 3 3 3 1 1 4 3 0 And I want to list only the unique values in column 'A' My desired df ['2'] I tried .drop_duplicates and .unique, ...

0
Q: How do you quantize a simple input using python

HEMSI am using the below codes to quantise the input signal for quantisation interval of 0.5 and this should give me staircase signal.The algorithm used here is same as used in Simulink.Could any one help me plot the quantised signal. import numpy as np import matplotlib.pyplot as plt for i in rang...

 
7:00 PM
0
Q: Why am I getting incorrect results from scipy.optimize.fmin?

Mateyobiimport pandas as pd from scipy.optimize import fmin data = pd.DataFrame({'DIV': [1,2,3]*3, 'MONTH': ['May','May','May','June','June','Jun','Jul','Jul','Jul'], 'C':[8]*9, 'U':[3,2,1]*3, 'S':[9]*9}) data.to_csv(r'...

0
Q: MatPlotLib Dollar Sign with Thousands Comma Tick Labels

Dance Party2Given the following bar chart: import numpy as np import matplotlib.pyplot as plt import pandas as pd df = pd.DataFrame({'A': ['A', 'B'], 'B': [1000,2000]}) fig, ax = plt.subplots(1, 1, figsize=(2, 2)) df.plot(kind='bar', x='A', y='B', align='center', width=.5, edgecolor='none', ...

0
Q: Coalesce values from 2 columns into a single column in a pandas dataframe

SevynsI'm looking for a method that behaves similarly to coalesce in T-SQL. I have 2 columns (column A and B) that are sparsely populated in a pandas dataframe. I'd like to create a new column using the following rules: If the value in column A is not null, use that value for the new column C If the ...

0
Q: How to execute UPDATE query on pandas dataframe?

user2212461If have a dataframe and want to update a column. I thought I could just use the same method that I also use for SELECT commands. Here's my code: disk_engine = create_engine('sqlite:///test.db') chunksize = 20000 j = 0 index_start = 1 for df in pd.read_csv('test.csv', chunksize=chunksize, ...

0
Q: My function is confused about whether or not it wants a second argument (python, matplotlib)

SteeeeeveI'm writing a (should be) simple program in Python using Tkinter and Matplotlib to control a machine. It's my first Python program, so apologies for any egregious non-conventions, and I welcome all feedback. Also, sorry for the large code block. I'm trying to have a main page for controls, and...

0
Q: How to adjust x limits in subplot without shrinking colorbar

CarolineI have a set of subplots that each require a colorbar. If I plot each subplot without setting the x limits, the x-axis extends past the domain of my data and shows a lot of white space. I'm using this code: import matplotlib.pyplot as plt from matplotlib.mlab import griddata from numpy import ma...

0
Q: pandas.read_csv not importing csv data

LMLPPNewb here trying to load csv data to do some manipulation and calculations. The read_csv function works, but when I call the Dataframe Variable, I only get about 9 of the 16 columns worth of data that are in the csv. Here is what I am using to import the data: temp = pandas.read_csv(directoryl...

0
Q: pandas.HDFStore.append_to_multiple with pandas.MultiIndex

KyleI am having a problem where I write a dataframe to two tables using pandas.HDFStore.append_to_multiple and then reading it back with pandas.HDFStore.select_as_multiple gives a dataframe with extra rows. df = pd.DataFrame(dict(a=[1,1,1,1,1,2,2,2,2,2], b=[4,4,4,4,4,4,4,4,4,4...

0
Q: Iterating over columns with for loops in pandas dataframe

Jeremy DroletI am trying to take a dataframe read in from CSV file, and generate scatter plots for each column within the dataframe. For example, I have read in the following with df=pandas.readcsv() Sample AMP ADP ATP 1A 239847 239084 987374 1B 245098 241210 988950 2A 238759 20...

 
8:04 PM
0
Q: Finding length of max discountinuity

leonsasI'm migrating some code from python list primitive to a pandas implementation. For some time series, I want to find all the discontinuous segments with their durations. Is there a clean way of doing this in pandas? My dataframe looks like this: In [23]: df Out[23]: 2016-07-01 05:35:00 60.466...

 
8:47 PM
0
Q: pandas module and Python 3 in OSX

loukiosI'm trying to make pandas module (that I've install through pip) working with Python 3.5. When I'm doing import pandas in Python 2.7 everything is fine, but I get the following error with 3.5 version: Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module...

 
8:57 PM
0
Q: First, small game with pygame

MaciejjyI have finished my first project for Python. I would be very gratefull if you could check it and give me some feedback. Just simple game with possibility of replay. Probably lots of mistakes and code could be half the size, but I just started with programming. :) I tried this game on few compute...

0
Q: Using Pandas' DatetimeIndex to facilitate creation of datetime column

Wirawan PurwantoI found that we can create date-time columns in a Pandas DataFrame by doing this: >>> dt1 = pandas.DatetimeIndex(["2016-03-04 15:01:49", "2016-03-05 23:54:22", "2016-04-03 21:22:08", "2016-04-03 21:22...

 
9:21 PM
0
Q: np.load to a paths file

badnerI have a file with all of my pathnames for each .npy file. I have about 5 million files so I would like to avoid unnecessary fors. What I need to do is to load them all into my data variable like this: data = np.load( input_file_w_pathnames ) I know this will not work but I was wondering if so...

 
9:42 PM
0
Q: Custom Matplotlib View Window

LukaszI've got a graph plotted using the matplotlib library, and would like to know if there is a way to view from a custom range. I have several vertical lines, starting 0. The beginning of the plot also starts at zero and makes seeing that first line difficult. Is there a way that I can have a view w...

0
Q: gensim: custom similarity measure

SimonUsing gensim, I want to calculate the similarity within a list of documents. This library is excellent at handling the amounts of data that I have got. The documents are all reduced to timestamps and I have got a function time_similarity to compare them. gensim however, uses the cosine similarity...

 
10:17 PM
0
Q: Force pandas xaxis datetime index using a specific format

user97662my panda dataframe looks like this: data1 data2 data3 DateTime .... 2016-04-18 16:16:53 -66 1 94.8654 2016-04-18 16:17:03 -67 1 94.8601 2016-04-18 16:17:13 -68 1 94.8410 2016-04...

0
Q: Create Pairs with respect to Date and Time

Sitz BlogzHow can I pair the two ids which fall in same date and time range. With same date and nearly same time (difference of 1 minute) in both connection StartTime and EndTime. Input Data: bd5a4d8305ee951499c12b9da67d8518,2010-03-08 17:07:32.012002,2010-03-08 17:09:18.917441 85c5acd19a304c4b209f7b9f...

 
10:44 PM
0
Q: Numpy csv file groupby

spicyramenI'm reading a CSV file with 2 columns, the second column describes a label, I want to see how many of each labels exists in my CSV file, I came up with a simple loop and a dictionary object. Wondering if there is a numpy function like groupby which gives me same functionality. dataset = np.genfr...

0
Q: ValueError: setting an array element with a sequence on reading multiple text files

LucasI have a folder containing 30 text files. What I need is to read them to create a dataset to train a sklearn SVM, but I'm getting the numpy error ValueError: setting an array element with a sequence. def loadDataset(dirName): dataset = [] files = listdir(dirName) print(files) ...

0
Q: C++ Pointer to Numpy Array

SamBriefly: Is there an efficient way to make a numpy array given a pointer in memory to the array, it's type, and the number of elements? More detail: I am working with a python framework which has an object.GetData() command that is supposed to return a pointer to the data (an array of 35,000 ...

 
11:14 PM
0
Q: numpy - incrementally rebuild an ndarray by filtering unneded members

Maxim HaytovichConsider that I have an ndarray: all_data.shape (220000, 28, 28) type(all_data) numpy.ndarray I want to go over each member of this array and filter out those which I don't want. As result I want to get new ndarray of exactly same shape. Something like: #save first image and its label in se...

 
11:34 PM
0
Q: How to optimize 2D data with leastsq in Python?

nxkryptorI have data of the form given below: X Y1 Y2 Y3 Y4 Y5 Y6 1.42857 4.83 4.58 4.43 4.31 4.22 4.14 1.40845 3.87 3.63 3.49 3.38 3.3 3.23 1.38889 3.17 2.93 2.79 2.69 2.62 2.56 1.36986 2.65 2.41 2.27 2.17 2.11 2.05 1.35135 2.27 2.02 ...

0
Q: Pandas subset setting values slow

Aleximport timeit import pandas as pd import numpy as np df = pd.DataFrame(np.random.rand(10, 10)) dft = df[[True, False] * 5] # df = dft dft2 = dft.copy() new_data = np.random.rand(5, 10) print(timeit.timeit('dft.loc[:, :] = new_data', setup='from __main__ import dft, new_data', number=100)) pri...

0
Q: What is the difference between native int type and the numpy.int types?

TheguyCan you please help understand what are the main differences (if any) between the native int type and the numpy.int32 or numpy.int64 types?

 
11:56 PM
0
Q: Seaborn, how do i add a reference line on a graph from my dataframe? "could not convert string to float"

Garrett EickelbergI have written this working script to iterate through my data frame and make a series of line plots for every gene name in my dataframe. What i would like to do now is have a vertical line on every plot that corresponds to a value in one of my dataframe columns. In my code below, the vertical lin...

 

« first day (860 days earlier)      last day (1328 days later) »