Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
Claims 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, 13, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Walker et al. (U.S. Patent Publication No. 2022/0362674), in view of Bell et al. (U.S. Patent Publication No. 2014/0043436).
Regarding claim 1, Walker discloses an information processing device, wherein the information processing device is configured to generate virtual space data that is available in a virtual space from scan information on an appearance of an object obtained by a three-dimensional scanner device (interpreted as the device must generate data usable in virtual space which comes from scan information representing the objects appearance)[Walker: 0011-0012 “According to a first aspect , disclosed herein is a method , implemented by a data processing system , for creating a virtual object in a virtual environment ; the method a comprising obtaining a digital representation of a real - world object , the digital representation representing a visual appearance of a shape of the real - world object”][Walker: 0016 “Consequently , one or more captured images of the real - world object , or another form of digital representation of the visual appearance of the real - world object , may be used by a data processing system executing the method as a basis for generating a virtual object having a user - defined appearance and attributes in a virtual environment”](teaches a data processing system that uses captured/digital appearance data of a real world object to generate a virtual object in a virtual environment), the three-dimensional scanner device is configured to obtain a three-dimensional shape and color information of the object [Walker: 0019 “The digital representation of the visual appearance of the real - world object may be a suitable representation indicative of a shape of the real - world object , e.g. a three dimensional shape of the real - world object . In particular , the digital representation may represent a point cloud and / or a surface geometry of the real - world object . The digital representation may further include information of visible properties of the real - world object , e.g. one or more colors , surface texture , and / or the like”](teaches the data includes 3D shape and properties like color), and the information processing device comprises: a determination circuit configured to determine a type of the object [Walker: 0111 “Generally , here and in the following the term processor is intended to comprise any circuit and / or device and / or system suitably adapted to perform the functions described herein”]; an obtaining circuit configured to obtain frame data, the frame data corresponding to the determined type of the object and being configured to map texture information obtained from the scan information (frame data is interpreted to mean structural model data onto which appearance/texture information can be mapped and in this context, frame data can reasonably include a skeleton template, model template, mesh, or virtual model structure corresponding to an object/character type)[Walker: 0029 “. Creating the visual representation may thus comprise creating the surface representation to define a shape and / or size / and / or color based on the detected shape and / or size and / or color of the real - world object and creating the skeleton to have a shape and size based on the detected shape and / or size of the real - world object . For example , creating the skeleton may comprise selecting one of a set of skeleton templates and a scaling the skeleton template based on the detected size and shape of the real - world object ; in some embodiments , a single template may suffice”](teaches obtaining/selecting skeleton templates corresponding to figure/object types and creates a surface representation containing shape/color/texture information and this skeleton/template and virtual model data structure are frame data and the scan derived color surface texture is applied to the virtual representation); but fails to explicitly disclose a display control circuit configured to display, on a display unit, a first screen including display of first scan information generated by scanning the object with the three-dimensional scanner device, the first screen being configured for adjusting a coordinate system set for the first scan information in accordance with a coordinate system set for the frame data; a reception interface configured to receive an instruction including a first operation instruction for at least one of movement and rotation with respect to the first scan information displayed on the first screen; and a first generation circuit configured to generate the virtual space data including texture information to be mapped on the frame data from the frame data and second scan information obtained after the display of the first screen ends in response to the instruction.
However, Bell discloses a display control circuit configured to display, on a display unit, a first screen including display of first scan information generated by scanning the object with the three-dimensional scanner device, the first screen being configured for adjusting a coordinate system set for the first scan information in accordance with a coordinate system set for the frame data (interpreted as the device must display a first screen showing first scan information and the screen is used to adjust the coordinate system of the scan data so that it corresponds to the coordinate system of the frame/template data)[Bell: 0061 “A primary 3D rendering 202 of successfully aligned captured 3D data 208 may also be shown. The display of this data allows the user to see what areas have thus far been captured.”][Bell: 0063 “The data 208 shown in 3D rendering 202 may include sets of points captured at various times by the 3D capture device with the different sets aligned into a common coordinate system for display”](teaches displaying captured 3d scan data and aligning sets of captured points into a common coordinate system for display); a reception interface configured to receive an instruction including a first operation instruction for at least one of movement and rotation with respect to the first scan information displayed on the first screen (interpreted device must receive a user input instruction to move and rotate the displayed first scan information, reception interface is interpreted as an input interface such as keyboard)[Bell: 0062 “The point of view may also be user-controlled thereby allowing the user to use touch, mouse, or keyboard input to change the point of view to browse various parts of the aligned captured 3D data. For example, in a touch inter face, a drag by a single finger may be used to rotate the 3D data. A pinch and spreading of two fingers may be used to Zoom out and Zoom in, respectively. A drag by two fingers may be used to move the viewpoint along the Surface of a horizontal plane”](teaches receiving keyboard/mouse input using gestures to rotate 3D data and move the viewpoint over displayed aligned captured 3D data); and a first generation circuit configured to generate the virtual space data including texture information to be mapped on the frame data from the frame data and second scan information obtained after the display of the first screen ends in response to the instruction (interpreted as the device must generate the final virtual space data, the generated data includes texture information mapped to frame data)[Bell: 0059 “The user may be directed to move or rotate the 3D capture hardware in a particular direction. If the current 3D scene alignment is known, instructions (e.g., “down,” “turn left, and the like) may be emitted to guide the user from the current position and orientation to a desired position and orientation”][Bell: 0063 “The data 208 shown in 3D rendering 202 may include sets of points captured at various times by the 3D capture device with the different sets aligned into a common coordinate system for display”](teaches user guided movement/rotation during scan alignment and multiple sets of captured 3D points aligned into a common coordinate system).
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of displaying captured 3D scan data and receiving user movement/rotation input to align the scan data in a common coordinate system. The motivation for such a combination would provide the benefit of improved scan alignment.
Regarding claim 2, Walker discloses the information processing device according to claim 1, but fails to explicitly disclose wherein the first screen includes display of the first scan information corresponding to any portion of the object and display showing a position of the portion in the frame data, and the reception interface receives an operation instruction for matching a display position of the first scan information of the portion with a display position of a position of the portion.
However, Bell discloses wherein the first screen includes display of the first scan information corresponding to any portion of the object and display showing a position of the portion in the frame data [Bell: 0060 “a user could be told to position the 3D capture hardware at or near part of an objector objects (e.g., “point the sensor at the back of the red couch')”][Bell: 0061 “A primary 3D rendering 202 of successfully aligned captured 3D data 208 may also be shown.”](teaches the display screen aspect and further teaches captured/aligned 3D scan information and can direct the user to a known area or object portion), and the reception interface receives an operation instruction for matching a display position of the first scan information of the portion with a display position of a position of the portion (interpreted as the input interface receives a user operation that aligns the displayed scanned portion with the displayed target reference position of that portion) [Bell: 0075 “unable to align a new captured 3D scene in a timely manner, the system may prompt the user to select a location, area, or previous capture position on the primary view 202 or map view 205 to indicate an area close to where the 3D scene has been captured”](teaches receiving user input on displayed 3D scan data to assist alignment. Selecting a location/area/previous capture position and rotating displayed 3D data are operation instructions used to bring scan data into alignment).
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of displaying a portion of captured 3D scan data and receiving user input to align that displayed scan portion with a corresponding displayed reference position. The motivation for such a combination would provide the benefit of more accurate matching between scanned object portions and corresponding positions in the virtual frame data.
Regarding claim 3, Walker discloses the information processing device according to claim 2, but fails to explicitly disclose wherein the first screen includes a switching operation interface configured to switch a display direction of the first scan information, and displays the first scan information of any portion of the object and the position of the portion in the frame data in a display direction corresponding to the selected switching operation interface.
However, Bell discloses wherein the first screen includes a switching operation interface configured to switch a display direction of the first scan information [Bell: 0062 “The point of view may also be user-controlled thereby allowing the user to use touch, mouse, or keyboard input to change the point of view to browse various parts of the aligned captured 3D data”][Bell: 0091 “Toggling between different types of data display or highlighting for the 3D view or 3D map view”](teaches a user interface that changes the point of view of aligned captured 3D scan data and includes controls for toggling rendering/display options for the 3D view/map view), and displays the first scan information of any portion of the object and the position of the portion in the frame data in a display direction corresponding to the selected switching operation interface (interpreted as after the user selects a display direction switch, the screen displays both the scanned portion and the corresponding reference/position in the frame data from that selected direction)[Bell: 0081 “This map view 205 may be displayed from various perspectives”][Bell: 0080 “the map view 205 may be an isometric, orthographic, or perspective 3D view of the 3D data captured so far (or some subset thereof).”][Bell: 0081 “In one embodiment, the 3D view is rendered from above, providing a top-down view of the data.”][Bell: 0063 “The data 208 shown in 3D rendering 202 may include sets of points captured at various times by the 3D capture device with the different sets aligned into a common coordinate system for display”](teaches displaying captured 3D scan data from selected perspective, including isometric, orthographic, perspective, and top down views, further teaches multiple captured point sets are aligned into a common coordinate system for display).
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of switching the display direction of captured 3D scan data during alignment. The motivation for such a combination would provide the benefit of allowing the user to view and align corresponding scanned portions and frame data positions from useful viewing directions.
Regarding claim 4, Walker discloses the information processing device according to claim 1, but fails to explicitly disclose wherein the first generation circuit generates the virtual space data so as to include position information of an image to be superimposed and displayed on the second scan information corresponding to a predetermined position of the object.
However, Bell discloses wherein the first generation circuit generates the virtual space data so as to include position information of an image to be superimposed and displayed on the second scan information corresponding to a predetermined position of the object (interpreted as the generated virtual space data must include location/position data for an image overlay) [Bell: 0076 “The primary view 202 may also allow users to select specific locations on the 3D data to add additional information”].
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of annotating a specific location on captured 3D data with an image. The motivation for such a combination would provide the benefit of allowing graphical images or overlays to be accurately displayed at selected positions on the generated virtual object.
Regarding claim 5, Walker discloses the information processing device according to claim 1, but fails to explicitly disclose wherein the display control circuit causes the display unit to further display a second screen for adjusting a position of an image to be superimposed and displayed on the first scan information corresponding to a predetermined location of the object, the reception interface is configured to further receive a second operation instruction for any one of movement and rotation with respect to the image displayed on the second screen, and the first generation circuit generates the virtual space data so as to further include position information of the image according to the second operation instruction.
However, Bell discloses wherein the display control circuit causes the display unit to further display a second screen for adjusting a position of an image to be superimposed and displayed on the first scan information corresponding to a predetermined location of the object [Bell: 0076 “The primary view 202 may also allow users to select specific locations on the 3D data to add additional information”], the reception interface is configured to further receive a second operation instruction for any one of movement and rotation with respect to the image displayed on the second screen [Bell: 0062 “the point of view may also be user-controlled thereby allowing the user to use touch, mouse, or keyboard input to change the point of view to browse various parts of the aligned captured 3D data”], and the first generation circuit generates the virtual space data so as to further include position information of the image according to the second operation instruction [Bell: 0077 “Annotating a specific location with text, image, or other data”].
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of annotating a specific location on captured 3D data with an image. The motivation for such a combination would provide the benefit of allowing graphical images or overlays to be accurately displayed at selected positions on the generated virtual object.
Regarding claim 6, Walker and Bell disclose the information processing device according to claim 4, wherein the predetermined location is a movable portion of the object [Walker: 0220 “The resulting second toy construction model thus comprises multiple parts which can be moved relative to each other where the available movements are defined by the skeleton elements 1132 , 1134 , 1133 and their respective interconnections 1135”].
Regarding claim 7, Walker and Bell disclose the information processing device according to claim 6, wherein when the object is a humanoid toy body, the movable portion is a joint portion [Walker: 0029 “the template skeleton may be defined such that the virtual object is animated so as to resemble a certain type of figure , such as a human - like figure having arms and legs and being animated to resemble a walking figure , or an animal having four legs , or a bird having wings and performing a flying movement , or a fish or snake - like figure being animated to perform a swimming or gliding movement”].
Regarding claim 10, Walker discloses the information processing device according to claim 1, but fails to explicitly disclose further comprising a position and posture control device provided with the three-dimensional scanner device and configured to control a position and a posture of the three-dimensional scanner device, wherein the first scan information is generated by the three-dimensional scanner device scanning the object at a predetermined position and posture provided by the position and posture control device.
However, Bell discloses further comprising a position and posture control device provided with the three-dimensional scanner device and configured to control a position and a posture of the three-dimensional scanner device [Bell: 0093 “The 3D capture hardware may be attached or coupled (either permanently or detachably) to any one of a variety of types of robots or other mechanized implementation rather than be manipulated by a human user”][Bell: 0099 “the robot or other mechanized implementation may rotate the 3D capture hard ware through a variety of orientations designed to cover a full 360 degree view of its surroundings”](teaches robot/mechanized implementation is a position control device because it moves and rotates the 3D capture hardware/scanner through different positions and orientations), wherein the first scan information is generated by the three-dimensional scanner device scanning the object at a predetermined position and posture provided by the position and posture control device [Bell: 0059 “If the current 3D scene alignment is known, instructions (e.g., “down,” “turn left, and the like) may be emitted to guide the user from the current position and orientation to a desired position and orientation.”][Bell: 0081 “the current or most recently known location 211 of the 3D capture device, the direction it is pointing, and/or its field of view may all be displayed in the map view 205”](teaches scanning from known current positions and orietntations and moving the scanner to desired positions/orientations).
Walker and Bell are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker to incorporate Bell’s teachings of controlling the position and posture of 3D capture hardware during scanning. The motivation for such a combination would provide the benefit of obtaining scan information from controlled scanner positions and orientations, thereby improving scan coverage and accuracy.
Regarding claim 11, Walker and Bell disclose the information processing device according to claim 1, further comprising a rotation device on which the object is placed and configured to rotate [Walker: 0133 “The system comprises a computer 101 or other data processing system , an input device 102 , a display 103 , a sensor device comprising a camera 104 , an object support comprising a turntable 105 , and a toy construction model 106 constructed from one or more toy construction elements of the toy construction system .”](teaches turntable rotates relative to the camera), wherein the first scan information is generated by the three-dimensional scanner device scanning the object placed on the rotation device [Walker: 0137 “To this end , a user may position the toy construction model 106 on the turntable 105”][Walker: 0146 “the process may capture a plurality of digital images of a toy construction model at respective angular positions of a turntable on which the toy construction model is positioned , or otherwise from respective viewpoints”](teaches scanning/capturing images of the toy model while it is positions on a turntable at different angular positions).
Regarding claim 12, Walker and Bell disclose the information processing device according to claim 1, wherein the determination circuit determines a type of the object based on information on a predetermined portion of the object [Walker: 0098 “recognizing a part of the virtual object as a representation of a recognized one of a plurality of predetermined parts , the recognized part corresponding to a subset of said plurality of geometric elements ; wherein digital representations of the plurality of predetermined parts are stored in a library of predetermined parts”][Walker: 0203 “It will be appreciated that the recognition of a subset of the toy construction elements of a toy construction model may be performed using a recognition process known as such in the art , e.g. by matching a 3D models of the known toy construction elements to parts of the obtained 3D model of the toy construction model and / or by the use of neural networks or other adaptive , data - driven techniques”](teaches recognizing object or model information by recognizing known predetermined parts/portions of the scanned model).
Regarding claim 13, Walker and Bell disclose the information processing device according to claim 12, wherein when the object is a humanoid toy body, the predetermined portion is a sole [Walker: 0029 “the template skeleton may be defined such that the virtual object is animated so as to resemble a certain type of figure , such as a human - like figure having arms and legs and being animated to resemble a walking figure”].
Claim 17 and 18 are non-transitory computer readable storage and device claims corresponding to claim 1 without any additional limitations. Thus, claims 17 and 18 are rejected for the same reasons as claim 1 above.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Walker et al. (U.S. Patent Publication No. 2022/0362674), in view of Bell et al. (U.S. Patent Publication No. 2014/0043436), in further view of Lee et al. (U.S. Patent Publication No. 2017/0211930).
Regarding claim 8, Walker and Bell disclose the information processing device according to claim 1, but fail to explicitly disclose further comprising a selection circuit configured to select, based on the determined type of the object, a fastener that defines a posture of the object when the three-dimensional scanner device scans the object.
However, Lee discloses further comprising a selection circuit configured to select, based on the determined type of the object, a fastener that defines a posture of the object when the three-dimensional scanner device scans the object [Lee: 0006 “there is disclosed a jig device for a 3D scanner, which fixes a chassis component for a vehicle and enables the chassis component to be scanned while rotating the chassis component”][Lee: 0028 “The turntable 10 has a flat support on which the target object 20 is put, and the Support is configured to be rotated at 360 degrees about a rotating shaft”](teaches using a jig/support/fixture in a 3D scanning system to hold/fix the target object for scanning).
Walker, Bell, and Lee are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker and Bell to incorporate Lee’s teachings of using a jig to fix an object in a defined posture. The motivation for such a combination would provide the benefit of allowing graphical images or overlays to be accurately displayed at selected positions on the generated virtual object.
Regarding claim 9, Walker and Bell disclose the information processing device according to claim 8, but fail to explicitly disclose wherein the first scan information is generated by the three-dimensional scanner device scanning the object in a posture determined by the fastener.
However, Lee discloses wherein the first scan information is generated by the three-dimensional scanner device scanning the object in a posture determined by the fastener [Lee: 0005 “a target object is put on a rotatable turntable and then scanned by a 3D scanner while the turntable is rotated at a full 360 degrees, and thus 3D scan data is produced”][Lee: 0006 “a jig device for a 3D scanner, which fixes a chassis component for a vehicle and enables the chassis component to be scanned while rotating the chassis component”](teaches scanning an object while it is fixed/supported by a jig or turntable).
Walker, Bell, and Lee are considered to be analogous to the claimed invention because they are in the same field of aligning scan data for virtual object creation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Walker and Bell to incorporate Lee’s teachings of using a jig to fix an object in a defined posture. The motivation for such a combination would provide the benefit of allowing graphical images or overlays to be accurately displayed at selected positions on the generated virtual object.
Allowable Subject Matter
Claims 14-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED TAHA whose telephone number is (571)272-6805. The examiner can normally be reached 8:30 am - 5 pm, Mon - Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571)272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED TAHA/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
Prosecution Insights