CSV Upload and Populate Module

thisisflashpoint · May 2011

I am wondering how one might go about creating a simple fuel module that would just have a form that would upload a csv file and then populate a database table with the data from that file. I know how to parse a csv file in php but am sketchy on how to setup the upload form since fields on the module forms generally map to the database table fields. Also, am sketchy on how once I have the rows from the csv table in array how to loop over the data check that's the row doesn't already exist and then insert it if it doesn't. Any thoughts or resources on how to approach this? Input would be much appreciated! Thank you in advance!

admin · May 2011

For this particular module, I'd create an "advanced module" like the seo, or validate modules, with it's own controller and view to display the form and process the data:
http://www.getfuelcms.com/user_guide/modules/advanced

With regards to the saving of the data, a FUEL models "save()" method uses INSERT IGNORE and then an ON DUPLICATE KEY UPDATE the values which essentially is similar to checking if a record exists and if it does, update it, else insert a new one (found in fuel/application/core/MY_DB_mysql_driver.php).

thisisflashpoint · May 2011

Thanks I will give this a shot and let you know how it goes.

thisisflashpoint · May 2011

A little way into this and things are going well ... one quick question I have come across which is probably easy and I am just not seeing it. On advanced modules, if I am grouping it like the blog or tools how do you get that group to show up in the left hand navigation above tools and manage but below site? Thanks again.

admin · May 2011

You would need to recreate the $config['nav'] in the order you want which is found on the fuel/modules/fuel/config/fuel.php config file. The ordering is just the natural order of the the associative array.

thisisflashpoint · May 2011

Got it thanks, I tried to create this in the My_fuel.php file but didn't seem to take so I just made the adjustment in the fuel.php file and that worked. Was that what you were recommending?

admin · May 2011

To do it in MY_fuel.php, you would essentially reset the variable, copy over what's currently in the main FUEL config and organize it like you want. MY_fuel.php get's included in at the bottom of fuel.php and will overwrite any similar values. However, if you don't reset the variable, it won't work correctly:

//MUST RESET THE VALUE HERE
$config['nav'] = array();

// site... Dashboard will always be there
$config['nav']['site'] = array(
	'dashboard' => lang('module_dashboard'),
	'pages' => lang('module_pages'),
	'blocks' => lang('module_blocks'),
	'navigation' => lang('module_navigation'),
	'assets' => lang('module_assets'),
	'sitevariables' => lang('module_sitevariables')
	);
// YOUR NEW CUSTOM MODULE
$config['nav']['my_custom_module'] = array();

// blog placeholder if it exists
$config['nav']['blog'] = array();

// my modules... if set to auto, then it will automatically include all in MY_fuel_modules.php
$config['nav']['modules'] = 'AUTO';

// tools
$config['nav']['tools'] = array();

// manage
$config['nav']['manage'] = array(
	'users' => lang('module_users'), 
	'permissions' => lang('module_permissions'),
	'manage/cache' => lang('module_manage_cache'), 
	'manage/activity' => lang('module_manage_activity')
	);

thisisflashpoint · May 2011

Got it that works perfectly ... thanks so much!

thisisflashpoint · May 2011

I have the csv files uploading and populating the database and everything is working great. Now I am going to create a front end interaction where a user can login and access the data. My question is concerning setting up the controller and views for the front end of the site. Do I also put these into the module directory and if so how do I configure the module to show up on the front end of the site I guess like the blog? Does that make sense? Thanks again in advance.

admin · May 2011

I would look at the blog as an example, in particular the fuel/blog/config/blog_routes.php file which maps the blog_module controller to the FUEL admin (note that it inherits the Fuel_base_controller) and the other controller do not.

thisisflashpoint · May 2011

Ok here is what I do know I have created an advanced module in a directory called members. I created a controller called members.php (code below) which I now see as an issue because when I put in someaddress.com/members in the browser that should be a CI_Controller and not the fuel_base_controller. So should I move this to be Members_module controller and save members as one my public front end controller? If so how does this change effect my configuration and setup? I don't understand the blog routes file particularly the piece below which may tie this all together. I understand the left side is just setting up the routes for the admin links like fuel/blog/posts ... what I don't get is the right side. Sorry I'm not seeing this ... any insight would be much appreciated!


$blog_controllers = array('posts', 'comments', 'categories', 'links', 'users');

foreach($blog_controllers as $c)
{
	$route[FUEL_ROUTE.'blog/'.$c] = FUEL_FOLDER.'/module';
	$route[FUEL_ROUTE.'blog/'.$c.'/(.*)'] = FUEL_FOLDER.'/module/$1';
}


require_once(FUEL_PATH.'libraries/Fuel_base_controller.php');

class Members extends Fuel_base_controller {
	
	public $view_location = 'members';
	
	function __construct()
	{
		parent::__construct();
		$this->load->config('members');
		$this->load->language('members');
		$this->_validate_user('members');
	}
	
	function _remap()
	{
		$vars = array();
		$vars['type'] = ($this->uri->segment(3) !== FALSE) ? $this->uri->segment(3) : 'names';
		
		if($this->input->post('upload'))
		{
			$this->_handle_upload($vars['type']);
		}

		$this->nav_selected = 'members/' . $vars['type'];
		$this->_render('members_upload', $vars);
	}

	function _handle_upload($type)
	{
		$config = array();
		$config['allowed_types'] = 'csv';
		$config['file_name'] = $type . '.csv';
		$config['overwrite'] = TRUE;
		$config['upload_path'] = MEMBERS_PATH . 'assets/csv';

		$this->load->library('upload', $config);

		if ( ! $this->upload->do_upload())
		{
			$this->session->set_flashdata('error', $this->upload->display_errors('', ''));
			redirect(fuel_uri('members/' . $type));
		}
		else
		{
			$upload_data = $this->upload->data();

			$this->load->module_library(MEMBERS_FOLDER, 'csv_reader');
         $csv_rows = $this->csv_reader->parse_file($upload_data['full_path']); 

			$model_variable = $type . '_model';
			$this->load->module_model(MEMBERS_FOLDER, $model_variable);
			$this->$model_variable->truncate();

			foreach($csv_rows as $csv_row)
			{
				$record = $this->$model_variable->create();
				$record->fill($csv_row);

				if (!$record->save())
				{
					$this->session->set_flashdata('error', lang('error_members_save'));
					redirect(fuel_uri('members/' . $type));
				}
			}

			$this->session->set_flashdata('success',  lang('success_members_save'));
			redirect(fuel_uri('members/' . $type));
		}
	}
}


/* End of file members.php */
/* Location: ./fuel/modules/members/controllers/members.php */

admin · May 2011

For the blog, the FUEL admin controller is called blog_module (for that reason in that I didn't want /blog/ to go to the FUEL admin).

With regards to the right side of the routes, if you create a 'simple' module, meaning one that is declared in you fuel/application/config/MY_fuel_modules.php file, then it gets routed to the generic FUEL 'module' controller. In the blogs case, each of those modules is a 'simple' sub module of the blog (essentially has a model) and thus we route it to the generic 'module' controller. The generic 'module' controller looks at the URI structure and grabs the correct initialization information from MY_fuel_modules.php to allow you to do the CRUD work for a module (or sub-module in this case). Does that make sense?

thisisflashpoint · May 2011

Yes that explanation did help me understand, thank you. I think what confused me is I couldn't find a mapping to the blog_module controller in the routes file. Seems like the blog_module is only being used for the dashboard? I think the connections that I made and finally helped were as follows

1. Controllers in an advanced module that extend CI_Controller get pulled up at the uri of the module folder name followed by / and the controller name unless the controller has the same name as the module folder then no controller name is needed.

2. The routes file for the module is for mapping the admin controller routes, front end routes happen by default. The default admin routes start with fuel/.

3. Simple modules can be used, all that is needed is a model and the routes to the module is mapped to the generic FUEL module like the blog routes file. If not using a simple module which I am not in this case set the route to point to that administrative controller.

Anyway, I add these in hopes they will help someone else. Basically what I have done now is moved my admin controller to another name different then the module folder name. I had made it the same before because of following the user guide and not grasping the above statements. Then I setup the routes in the route file for the admin to map to the changed controller name. Now I am creating my controllers for the front end of the web site starting with one that does have the same name as the directory. Anyway, seems like small revelations but they paramount for me. Thanks again for the help!

admin · May 2011

Thanks for the clarifications above for others. I'd also add that looking at the fuel/modules/config/fuel_routes.php file may help understand the routes some more because it contains the basic routing logic (comments in file should help explain).

thisisflashpoint · May 2011

Ran into an item and wondering if its a bug or how its supposed to work. I was wanting to setup cms pages that map to the address of module's controllers. Like a page with a location of members and a page with a location of members/change_password. So the client can administer the content. I am explicitly setting the location like so


$config = array();
$config['location'] = 'members/change_password';
$config['render_mode'] = 'auto';

$this->fuel_page->initialize($config);

when attempting to render the pages. This worked fine for the members default controller it loaded the page as expected but it didn't work for members/change_password. I can only seem to get it to work if I make the page location change_password without members/ on the front it. Does this make sense and is it possible to do what I want to do?

admin · May 2011

A couple questions:
1. Is the change_password view file located at fuel/application/views/members/change_password? Or are the view files in a different module's view folder (e.g. fuel/modules/members/views)?
2. What is showing up right now?... anything?

thisisflashpoint · May 2011

I'm not using a view file I am trying to use db content mapped to the location members/change_password. If I just use a location of change_password it pulls the db information just fine but when I try to us members/change password as the location it doesn't work I don't get any html. I am changing the location in the admin when change the in my config variable. From what I can tell its not getting the page_data when I try to use members/change_password in the fuel page object and therefore nothing set as the layout. The layout file is stored in fuel/application/views/_layouts.

thisisflashpoint · May 2011

Can't track it down but it would seem that if I set a location and render_mode to auto on a Fuel_page object on initialization that it would ignore the uri and use the location to load from the database. Is that how its supposed to work or do I misunderstand that?

admin · May 2011

I think I may see your issue. In fuel/modules/fuel/libraries/Fuel_page.php file, in the _assign_location() method around line 90, it grabs the segments directly from the URI and it ignores what is in the location value if you are pulling from the database. What happens if you change that code to the following (I have it tested locally and it seems to work OK):
CHANGE:
$segments = $this->_CI->uri->rsegment_array();

TO:

if (!empty($this->location))
{
	$segs = explode('/', $this->location);
	$segments = array_combine(range(1, count($segs)), array_values($segs));
}
else
{
	$segments = $this->_CI->uri->rsegment_array();
}

thisisflashpoint · May 2011

Yep that does the trick, thank you! Definitely helps when developing an application that is tied in with a regular web site. Allows the client to admin the pages of the custom areas as well.

admin · May 2011

I agree... I'm pushing a fix for that very soon as well as an issue with the editor being used in inline editing showing a javascript error (so you may want to do a git pull).

thisisflashpoint · May 2011

Sounds great ... thanks for the heads up. I have a client who is using the ckeditor and reported the following "To get a hyperlink to stick when using the page editor, you have to add the hyperlink and then view source. If you do not view source before hitting save, the link disappears." I haven't tried to replicate this yet but will try to do so and let you know what I find out. If there is an issue I will put in a bug post.

admin · May 2011

ok... thanks for the heads up. I'll see if I can replicate on my end as well.

Howdy, Stranger!

Categories

CSV Upload and Populate Module

Comments