Models for key value pair extraction in NLP

There are a range of models, articles and information available which extract keywords from unstructured documents. Examples are Spacy, POS tagging, NER, entity extraction etc What I am looking for is key value pair extraction and not able to find much relevant articles or models or even datasets. Few examples of key value pairs are: Name : Sandeep Company Name: Sandeep Total Amount $10000 Invoice INV110011 or INV | INV110011 There can be a range of formats in which key …
Topic: nlp
Category: Data Science

How do I calculate a similarity matrix with a Student-t kernel?

As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix for that. A few steps before the Q-matrix, I need the similarity matrices made using the un-normalized Student-t kernel. I'm using r, not sure if that's relevant to an answer.
Category: Data Science

Calculate days for each years between two dates in a pandas dataframe

I have a usecase wherein the from and to dates are provided. I need to calculate the total_days in each year and the total orders for each year. I'm currently coding in python Below is an example OrderNo From_dt To_dt 1 6/11/2017 7/8/2019 2 06/02/2018 10/02/2018 Target Data that I want to achieve: Year Total_Days Total_Orders 2017 300 20 2018 200 10 2019 400 70 I need to dynamically create the years like in the first example 2018 will have …
Category: Data Science

add_action on inherit post status

I'm trying to use the add_action() hook to run a custom function, but am struggling with the post status. I originally tried using: add_action('pending_to_publish_portfolio', 'my_function'); (portfolio being my custom post type). This didn't work, so i posted in the official Wordpress Support forum for some help. I was given the following code to get the right transition status' function bj_test($new, $old) { print_r($old . '_to_' . $new); echo '<br>'; print_r($old . '_' . $new); die(); } add_action('transition_post_status', 'bj_test', 10, 2); …
Category: Web

Sort posts by child category name

I have a category "year" and child-category names with year numbers 1966, 1972, 1980 etc. I have a category "publisher" and child-categories with publisher names. Each post is assigned a year child-category and a publisher child-category. When listing posts with child-category of a specific publisher, I want to order them according to the name of the child category of category "year". I looked around and didn't find a solution. Thanks
Category: Web

Dimensionality of CNN and a linear function

Hi I need help understanding how the nn.linear can be implemented in a neural network without problems dimensionality. EDIT I think my problem relates to understanding the relationship between in and output channel. Why does this work: m2 = nn.Linear(32, 31) input = torch.randn(128, 32) output = m2(input) print(output.size()) When this doesn't work: m2 = nn.Linear(512, 4) input = torch.randn(128, 2, 2, 4) output = m2(input) print(output.size()) Considering that the input of Linear matches the three first dimensions of the …
Category: Data Science

How to remove the message 'We could not find any results for your search' without changing template files and without adding posts/pages?

I take the question back with context. The goal is not to add posts or pages so that the post is replaced by the title of the post/page. The goal is not to modify the search.php file to remove this error message by hand. Imagine the situation. We have a normal page with a search widget in a sidebar or in the footer. If you launch the search, you will have your site loading due to the widget submit. Wordpress …
Category: Web

Removing seasonality in time series forecasting

In time series forecasting we are removing the "seasonal" component to fit models better and have better forecasting. But why? if I should give an extreme example: if I have a sin wave, I wouldn't remove the seasonal component of it. Because it is much easier to forecast with its own structure (because it has an easily recognizable pattern). Am I wrong?
Category: Data Science

Training using a dataset on NAS server

I have 2 questions: how would you approach the storage of datasets (with millions of small files) on local network? And how would you take that into account in pytorch code? Hello, I need to store large datasets (could be in the TB) on local network. However, training using a dataset on the 3 different NAS servers I tested on was consistently 4 time longer while GPU usage was 25% on average, I guess that's because the GPU isn't fed …
Category: Data Science

AJAX call inside plugin class is not getting to it's response function

I've looked at a heap of other people with this same problem but I still can't find a solution. Hopefully someone can see what I've missed because I'm about to pull my hair out. My ajax call is not getting to the response function. This is how my code is called. This function gets called from a template. function build_a_boat(){ // blah blah blah require_once BUILDNEWBOAT_PATH . 'includes/BuildNewBoatPlugin.php'; $build_boat_plugin = new BuildNewBoatPlugin(); $build_boat_plugin->run(); } The below code is inside by …
Category: Web

Multisite wp-admin redirect loop

I have a subdirectory multisite setup, it all works fine apart from one small issue... The sites are setup in the following structure: http://domain.com/blogs/ http://domain.com/blogs/second/ http://domain.com/blogs/third/ You can access the admin panels if you use the following: http://domain.com/blogs/wp-admin/ http://domain.com/blogs/second/wp-admin/index.php http://domain.com/blogs/third/wp-admin/index.php But if you use the following (as wordpress does in the dashboard), then it gives a redirect loop? http://domain.com/blogs/second/wp-admin/ http://domain.com/blogs/third/wp-admin/ It seems there is some sort of redirect just when index.php is not specified on the subfolder installs, what is …
Category: Web

Kmeans clustering in python - Giving original labels to predicted clusters

I have a dataset with 7 labels in the target variable. X = data.drop('target', axis=1) Y = data['target'] Y.unique() array(['Normal_Weight', 'Overweight_Level_I', 'Overweight_Level_II', 'Obesity_Type_I', 'Insufficient_Weight', 'Obesity_Type_II', 'Obesity_Type_III'], dtype=object) km = KMeans(n_clusters=7, init="k-means++", random_state=300) km.fit_predict(X) np.unique(km.labels_) array([0, 1, 2, 3, 4, 5, 6]) After performing the KMean clustering algorithm with a number of clusters as 7, the resulted clusters are labelled as 0,1,2,3,4,5,6. But how to know which real label matches the predicted label. In other words, I want to know how …
Category: Data Science

Displaying page content from plugin, inside exising empty WP page

Old php developer here, very.. very new to WP, so be kind. I am building my first WP Plugin. I have this in the plugin central file (the one that is /plugins/my_plugin/my_plugin.php): // Setup the admin management page function wx_admin_burger_management() { add_menu_page( 'Burger Manager', // Page title 'Burger Manager', // Menu title 'manage_options', // Capability 'admin-manage-burger', // Menu slug 'wx_burger_admin', // Callback function 'dashicons-list-view', // Menu icon '3' // Priority ); } function wx_burger_admin() { include(trailingslashit(WX_PLUGIN_PATH).'my-burger-admin.php'); } add_action('admin_menu', 'wx_admin_burger_management'); This …
Category: Web

Successive Predictions

I am facing a problem which you could abstractly describe the following: I have a pool of possible customers. I want to know if a customer appears during one year. I want to predict the total revenue for each customer for this year. (0 revenue is something different than not appearing) So I have stacked/successive/dependent predictions: I predict who out of the pool is going to appear For these I predict what the revenue will be So far so good. …
Category: Data Science

Model Guardrails

Suppose I am building a machine learning model for an application where I do not need to make a prediction on all new samples, and given a new sample, it is better to make no prediction at all when there is concern that the prediction is unlikely to be good (for example, if the new sample appears to be very different than the training samples). I'm calling the idea of restricting which new samples to make a prediction on "model …
Category: Data Science

I have cloudflare flexible enabled, How to redirect to https for Nginx server

If I access directly to https it works, but the site is still accessible by HTTP. How do I force all http to https? I have used WordPress HTTPS plugin. But it is not redirect to https. I have configured the nginx with following: server { listen xxx.x.xxx.xxx:80; listen 127.0.0.1:80; server_name mydomain.com; root /var/www/html/mydomian.com/; index index.html index.htm index.php; charset utf-8; location / { try_files $uri $uri/ /index.php?$args; rewrite ^(.*)$ https://$http_host$request_uri redirect; } ... But this redirection will leads to infinite …
Category: Web

A specific menu inside a page, load different contents without reloading it, under the same link

I'm developing a website for a NGO, and they need it as user-friendly as possible since it's tailored towards folks over 65. So the challenge here seems a little big. I just have way too many questions on how to go about this. They will need a page that has a menu inside it (tabs), and each tab has a different content under it, and this menu is specific for the page (it won't show up in any other part …
Category: Web

Best practices for scoring hundreds of models on the same massive dataset?

I have 500+ models predicting various things and a massive database of over 400m+ individuals and about 5,000 possible independent variables. Currently, my scoring process takes about 5 days, and operates by chunking up the 400m+ records into 100k-person pieces and spinning up n-number of threads, each with a particular subset of the 500+ models, and running this way until all records are scored for all models. Each thread is a Python process which submits R code (i.e. loads an …
Topic: scoring
Category: Data Science

Using wp_tag_cloud with only current post tag with special class

I am trying to output all the tags (custom taxonomy) without link, and add special class to current post tags. For example: if there are 10 tags but current post has 3 of them applied, then it should display all the 10 tags without link but only 3 tags should have special class. I am currently using: $terms = get_the_terms( $post->ID, $tax ); $tag_list = implode(',', wp_list_pluck($terms, 'term_id') ); wp_tag_cloud( array( 'taxonomy' => $tax, 'include' => $tag_list) ); which is …
Category: Web

For loop over a tensor in TensorFlow?

I have a tensor of shape (1, M) where M is a multiple of 10. When I print the tensor it may look something like Tensor("some_name", shape = (1, 80), dtype = float32) This tensor is the output of a neural network which will be run in a session. I wanted to modify this tensor according to the following (broken)code: for chunk_number in range(int(tensor.shape[1]/10)): this_chunk = my_fn(tensor[10*chunk_number : 10*(chunk_number+1)]) // my_fn is custom tf.assign(tensor[10*chunk_number : 10*(chunk_number+1)], this_chunk) This is essentially …
Topic: tensorflow
Category: Data Science

Generate one time URL

I need to implement one time URLs in a wordpress website. I found this script on phpclasses website and I integrated it in wordpress and it seems to work. I think this script can also be useful to other users, and so I'd like to ask if someone can help me. In my case, the script is rewriting the URL, but when I click on it, it redirects me to a 404 page. Is it possible that the problem is …
Topic: urls Wordpress
Category: Web

How to extract the sample split (values) of decision tree leaves ( terminal nodes) applying h2o library

Sorry for a long story, but it is a long story. :) I am using the h2o library for Python to build a decision tree and to extract the decision rules out of it. I am using some data for training where labels get TRUE and FALSE values. My final goal is to extract the significant path (leaf) of the tree where the number of TRUE cases significantly exceeds that of FALSE ones. treemodel=H2OGradientBoostingEstimator(ntrees = 3, max_depth = maxDepth, distribution="bernoulli") …
Category: Data Science

Rewrite parameters as Url for SEO

I have urls of this form: example.com/event/?eventId=1 example.com/event/?eventId=2 The problem is Google is indexing only example.com/event, which is a blank page. It's not able to display an event without the Id. I would like to redirect this url to this form: example.com/event/1 This will allow Google to index each event individually. The event page is provided by the admin. I imagine something like: function add_eventId_rule() { $post = get_post(get_option('EVENT_AGENT_EVENT_PAGEID')); if($post != null) { add_rewrite_rule( substr(wp_make_link_relative(get_permalink($post)), 1) . '(\d*)/?$', 'index.php?page_id=' . …
Category: Web

Time series on syslogs

Is time series model suitable for network syslogs considering the fact the messages are sequential and the messages are outputted as a result of dependency between themselves which can range from normal notifications to alarms if there are. Can the state of the art models like ARIMA be applied ? What kind of questions can be answered with applying time series models on syslogs ?
Topic: time-series
Category: Data Science

How can I click the parent menu then redirect to a special URL?

You see I have a Navigation bar items, in the parent item Products & Services with two sub menus. I have a requirement, when I click the menu item Products & Services it redirect to its sub-menu Dedicated Servers. How can I do with this? EDIT-01 I only want the Products & Services parent have this redirect, the other parent do not need, such as Home, Advantage they all have their own page.
Category: Web

Show custom default avatar instead of gravatar

I'm using this custom function to get the default avatar from my server instead of gravatar: if(!function_exists('custom_avatar')){ function custom_avatar($avatar_defaults){ $new_default_icon = 'http://localhost/gv/wp-content/images/mystery-man.png'; $avatar_defaults[$new_default_icon] = 'Custom Avatar'; return $avatar_defaults; } add_filter('avatar_defaults','custom_avatar'); } but the custom avatar is not showing up, when I view source code then the src of the image look like this: http://0.gravatar.com/avatar/a432e8915b383edd8d25c2a4fd5a6995?s=32&d=http%3A%2F%2Flocalhost%2Fgv%2Fwp-content%2Fimages%2Fmystery-man.png%3Fs%3D32&r=G&forcedefault=1 Why does my image src relative to gravatar here? How can I fix this problem?
Category: Web

Sensitivity analysis in outlier explanation

I am trying to find the outlier explanation using the sensitivity analysis. Let’s consider that my dataset contains 19 different input values and 1 output value (So overall 20 different columns are there and values are numerical). I have already made a prediction model and I am considering the values with high prediction errors are outliers/ anomalies. I have done the sensitivity analysis for individual input values but in the dataset values are correlated with some other input values, e.g. …
Category: Data Science

Programatically upload a file to be stored inside blob field in database, NOT on filesystem

old php developer here, very.. very new to WP, so be kind. As part of a plugin, I am trying to allow files to be uploaded with the intended destination storage in a specific table. I have the following code which works fine (almost). // File handing preparation. Count how many files were uploaded $wx_countFiles = count($_FILES['file']['name']); // For each uploaded file, process them accordingly for($i=0; $i<$wx_countFiles; $i++) { // Assign values required $wx_fileName = $_FILES['file']['name'][$i]; $wx_fileMime = $_FILES['file']['type'][$i]; $wx_fileSize …
Category: Web

Duplicate notifications for new user registration

I am trying to figure out why new user registrations are receiving duplicate notification emails. They arrive at the exact same time but from different addresses, one is from the admin owner (myself) and the other one from [email protected]. I am using a custom registration form. Here are the relevant pieces of code: // auto generate a password $user_pass = wp_generate_password(); // setup new user $userdata = array( 'user_pass' => $user_pass, 'user_login' => esc_attr( $_POST['user_name'] ), 'user_email' => esc_attr( $_POST['email'] …
Category: Web

How to make use of POS tags as useful features for a NaiveBayesClassifier for sentiment analysis?

I'm doing sentiment analysis on a twitter dataset (problem link). I have extracted the POS tags from the tweets and created tfidf vectors from the POS tags and used them as a feature (got accuracy of 65%). But I think, we can achieve a lot more with POS tags since they help to distinguish how a word is being used within the scope of a phrase. The model I'm training is MultnomialNB(). The problem I'm trying to solve is to …
Category: Data Science

How to make an ensemble model for classification with pytorch using trained models?

I am trying to make an ensemble model composed of two pre-trained models, using torch, in order to classify an image. Below is some code, based on this post. import timm import torch from torch.nn import functional as F num_classes = 100 model1 = timm.create_model("efficientnet_b0", num_classes=num_classes) checkpoint1 = torch.load(checkpoint_path1) model1.load_state_dict(checkpoint1["model"]) model2 = timm.create_model("efficientnet_b2", num_classes=num_classes) checkpoint2 = torch.load(checkpoint_path2) model2.load_state_dict(checkpoint2["model"]) class EnsembleModel(nn.Module): def __init__(self, modelA, modelB, num_features): super().__init__() self.modelA = modelA self.modelB = modelB self.classifier = nn.Linear(2*num_features, num_features) def forward(self, x): x1 …
Category: Data Science

How can i merge two datasets with similar words in python?

For instance i have a row value on the dataset_1 : "Entity" = Apple dataset_2 : "Entity" = iCloud Apple (Entity is the column) I need to merge one dataset to the other by the column entity, but to do that i need them to have exacly the same value and Apple ≠ iCloud Apple. Both datasets are huge so i cant do this manually, one by one
Category: Data Science

How to deal with data having 0 values in many columns?

I am trying to implement logistic regression but the dataset that I have have many columns with skewed data and most of them have 0 as values. I also the skewness of data for many columns its going above 190. But it's not only for training data, it's the same for testing data too. I tried using log method to remove skewness but because most of the value is 0 it messed up my data. I don't know how to …
Category: Data Science

Add active class to foundation 6 tabs while looping categories

I have a custom post type (bulletins) that I'm putting into Foundation 6.4.3's tabs and accordions. The taxonomies are the tab headings. The individual custom post type content is the accordions. My problem is on page load, no tabs appear. I need to add 'is-active' class to the first class="tabs-panel" on the page, but I can't figure the best way to do that because I have to start and stop the various loops to get the tabs and accordions to …
Category: Web

how to remove wrapping widget

I want to remove the default wrapping ul and li class="page_item page-item-2" on the widget this is my code register_sidebar([ "id" => "footer1", "name" => __("Footer 1"), "description" => __("A short description of the footer."), 'before_widget' => '<ul>', 'after_widget' => '</ul>', 'before_title' => ' <li><span class="footer-links-title">', 'after_title' => '</span></li>', ]); I want the result like this: <ul> <li><span class="footer-links-title">Information</span></li> <li><i class="fa fa-chevron-circle-right"></i> <a href="" class="links">Terms & Conditions</a></li> </ul> when I try it the result is like this: <ul> <li> <span …
Category: Web

How to remove categories filter from wordpress admin?

I removed the categories column from "All posts" page by applying this code. add_filter("manage_edit-post_columns", "my_post_edit_columns"); function my_post_edit_columns($columns){ unset($columns['categories']); return $columns; } This code removed categories column. But still i see the categories filter in the top. Is there a way to remove it other than using CSS to hide it? Thanks
Category: Web

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.